Asteroid Detection Module¶

Lecturer: Quanzhi Ye
Jupyter Notebook Authors: Quanzhi Ye, Ashish Mahabal, Dmitry Duev, & Cameron Hummels

This is a Jupyter notebook lesson taken from the GROWTH Summer School 2019. For other lessons and their accompanying lectures, please see: http://growth.caltech.edu/growth-astro-school-2019-resources.html

Objective¶

Compute the detection rate of fast-moving asteroids, asteroids that pass within ~20 lunar distances from Earth, and leave trails/streaks on astronomical images.

Key steps¶

• Figure out which known asteroids will show up as streaks in ZTF images, and
• Examine the streak catalog to see how many of them are detected.
• [Optional] Run DeepStreaks, is a convolutional-neural-network, deep-learning system designed to efficiently identify streaking fast-moving near-Earth objects detected in the ZTF data

Required dependencies¶

See GROWTH school webpage for detailed instructions on how to install these modules and packages. Nominally, you should be able to install the python modules with pip install <module>. The external astromatic packages are easiest installed using package managers (e.g., rpm, apt-get).

Python modules¶

• python 3
• astropy
• numpy
• pandas
• astroquery
• matplotlib
• pillow

External packages¶

None

If you are missing dependencies, pip-install them from this notebook:

Step 1: fast-moving asteroids that fall on ZTF images¶

We will loop over the catalog of known NEAs (Near-Earth Asteroids) and make a series of queries to JPL's Horizon service and IPAC's IRSA service, to determine if and when they will be visible at Palomar (where ZTF runs from).

Let's download the catalog of known NEAs from the Minor Planet Center (MPC).

Let's convert the catalog into a pandas DataFrame for convenience. The format of the MPC catalog is described here, it's best for machines but is also somewhat human readable.

Now let us loop over the MPC catalog to check the visibility for every NEA. We will do it for NEA 2018 CL (the first NEA found by ZTF), and it's up to you to write a loop for it.

First, we create an instance for the Horizons query. We only need to tell Horizons the name of the object we want to query, and the time window we want to query, and let Horizons do the heavy-lifting. Oh, and the three-letter-code for ZTF is I41, as described here. If the query returns zero entries, it means that the object is not observable at Palomar at the suggested time (interval).

First, let us define a time window we would like to query for. Our default is 20180205, the day ZTF science observations started:

Then let's do all the tasks described above. For now, we do our test with 2018 CL (the first NEA found by ZTF).

We also want to calculate the motion of the asteroid to see if it will show up as a streak in our images.

Now, since we just discovered the power of JPL Horizons... why don't we take a short detour and do something fun? Let's say, we want to see the predicted trajectory of 2018 CL...

What about the change of azimuth and elevation angle?

The complete lists of the values supported by astroquery.jplhorizons can be found here. Play with it and be creative! For example, can you plot the change of RA/DEC rate over time? And what about the change of predicted V magnitude?

Now let's come back to our main journey. How fast does the asteroid need to be in order to show up as a streak? Let's take anything longer than 5 full-width-half-maximum (FWHM) as a streak. We know the exposure time of ZTF is 30 seconds, and the typical FWHM is 2''. Therefore, the minimum rate of motion is not too difficult to calculate:

If the object will be a fast-mover at some point within the time window being queried, we will use IPAC's Moving Object Search Tool (MOST) to get a list of the images that the object is a fast-mover on them.

...and MOST did give us something, right?

Here is a loop over the query result from MOST to find out which images will contain the target as a fast-mover. Note that we want the predicted magnitude of the object to be <20 since that's the typical depth of ZTF.

We also make another query to Horizons to ensure the object is really a fast-mover on these dates. We do this as a double-check since Horizons produces the most precise ephemerides for NEAs. Horizons also tells us what the expected positional uncertainty will be. Obviously, we don't want the positional uncertainty to be too big -- say, over 20".

And then we print out all the metadata that are potentially useful. We will just return the first entry (or it will be a very long entry!)

Voila! You made it. Note that it takes quite a bit of time (a day or two) to loop over the entire NEA catalog -- this is because Horizons has to calculate the ephemerides for each of these ~15,000 NEAs and this is slow! Therefore, we provide a pre-generated result file for you to proceed to step 2.

But do take a few known asteroids and see what you get for them.

Step 2: compare the catalog derived from step 1 to the source catalog generated by ZTF streak detection pipeline¶

Let's read in the result of step 1... (OK, we cheated, it is pre-generated)

And also we read the source files from ZTF... essentially, these are a large number of "streak_qa" files, each file contains candidate streaks in one image. Apparently, we want to loop over them and see if there is any candidate that coincides the position of a known streaking NEA. We also record the rate and mag of detections and non-detections for diagnostic purposes.

For a demonstration, let us pick one of them to show how this whole thing works. Let us start with our old friend, 2018 CL.

What are the uncertainty ranges (in arcsec) for these entries?

This is very precise! Well, we know it should be small because the orbit from JPL was calculated using ZTF observations :)

Now, let us obtain the predicted coordinates for 2018 CL at these epochs using what we have just learned in Step 1.

A simplistic approach is to loop through each small folder generated by findStreaks and see if there is any candidate matching the predicted RA and DEC:

Wonderful! We have a hit! We can even take a look at the cutout to see what it looks like:

Voila! Now, homework for you: can you use the example shown above to work out the detection efficiency of ZTF?

Step 3: run a light version of DeepStreaks [optional]¶

DeepStreaks is a convolutional-neural-network, deep-learning system designed to efficiently identify streaking fast-moving near-Earth objects, detected in the ZTF data.

For details, please see Duev et al., MNRAS.486.4158D, 2019, arXiv:1904.05920, and this Github repo.

We will load a light version of DeepStreaks and run it on the streaks to see if they would have been identified. DeepStreaks is implemented using Google's TensorFlow library, so we will need it for this part.

DeepStreaks consists of three sets of binary classifiers: the first one (real/bogus or $rb$) decides if a particular cutout contains a steak-like object (which includes the actual streaks and cosmic rays, for example), the second (short/long or $sl$) decides if a streak is long (i.e. caused by something that moves too fast like a LEO satellite) or short, and the third (keep/ditch or $kd$) makes the final judgement. If all three parts output a score >0.5, a streak is designated as a plausible FMO candidate.

Download models from Github if necessary:

Load models:

Now let us load the cutout image and resize it to the shape expected by the models.

Great! This streak would have been successfully identified by DeepStreaks!

Run DeepStreaks on some random streaks¶