Introduction to Footprints¶

Owner(s): Imran Hasan (@ih64)
Level: Intermediate
Last Verified to Run: 2021-04-01
Verified Stack Release: 21.0.0

Learning Objectives:¶

After working through this tutorial you should be able to:

1. Understand footprints, and their relationship to detection, and measurement
2. Use methods and attributes from the footprint class to get information like centroids, size, shape, of the footprint
3. Perform operations on footprints, like dilation an errosion
4. Understand the relaitonship between peaks and footprints, and their role in deblending
5. Use footprints to access pixel level data in their parent exposures

Logistics¶

This notebook is intended to be runnable on lsst-lsp-stable.ncsa.illinois.edu from a local git clone of https://github.com/LSSTScienceCollaborations/StackClub.

When editing the stackclub package files, we want the latest version to be imported when we re-run the import command. To enable this, we need the %autoreload magic command.

You can find the Stack version that this notebook is running by using eups list -s on the terminal command line:

For this tutorial we'll need the following modules:

Footprints¶

To paraphrase from Bosch et al. (2017),

Footprints record the exact above-threshold detection region on a CCD. These are similar to SExtractor’s “segmentation map", in that they identify which pixels belong to which detected objects

As you might expect, this means footprints are integral to high-level CCD processing tasks—like detection, measurement, and deblending—which directly impact science results. Because footprints are so closely related to these very important processes, we will take a close look at them in this notebook.

The Basics: Footprint Geometry and Location¶

In the quote above, an analogy was drawn between footprints and segmentation maps, as they both identify above threshold pixels. As we first introduce footprints, we will concentrate on this similarity as it gives us a place to start understanding the location and geometeric properties of footprints.

We will use the detectFootprints method in SourceDetectionTask to find and store the detected footprints in the image

detect_res is a Struct. Struct objects are similar to python dictionaries, in that they have key value pairs. However, here you access the values by using the keys as an attribute. If you are and IDL user, these will remind you of Structures in IDL.

Looking at detect_res, we can see that there are attributes associated with sources detected positive (above threshold) and detected negative (below threshold). Let's concentrate on the positive detections for now, so we can look at astrophysical sources. The positive attribute holds a FootprintSet, an object that contains all the positively detected footprints. The numPos attribute tells us the number of footprints that were detected positive. The numPosPeaks attribute tells us the number of peaks that were detected positive. You'll notice that the number of peaks is larger because more than one peak can belong to a footprint. We will discuss peaks more fully in a later section, but for now think of them as what they sound like—the peaks in the surface brightness profile.

the variable fpset is a FootprintSet object. While it contains many footprints itself, a few more steps are required to obtain individual footprints. Namely, we will ask for a FootprintSet to give us a list of footprints, which we can iterate over

Now we have drilled our way down to getting our hands on footprints. Lets discuss some of their methods and attributes. To begin, we will concentrate on footprint geometry and location.

The bounding box of the footprint. This is the smallest rectangle you can enscribe around the footprint. Not all of the pixels in the bounding box are pixels in the footprint, since the footprint is not necessarily rectangular in shape. the BBox will always be inside the region

Get the area of the footprint. NOTE: this is not the area of the bounding box. It is the number of pixels associated with the footprint.

The shape of a footprint can be calculated. It is assumed to be an ellipse, and the relevent parameters are defined with the quadurpole, $Q_{i,j}$, a matrix containing the second moments, where

Notice this definition does not include the surface brightness of the detected object. That is to say, the quadropole computed for footprints describes the geometry of the pixels in the footprint, not the intensity weighted shape of its detected object.

The elements for a footprint can be computed with the getShape() method. Note that the matix is symmetric.

Looking at $Q_{i,j}$, the diagonal elements are close in magnitude, and the off diagonal elements are close to zero. We expect the footprint to be roughly circular. Now that we have all the pertinent information about where the footprint is and what pixels belong to it, we can display it and see if our expectation is true.

Visualizing Footprints & Heavy Footprints¶

We get information about which pixels in the footprints bounding box actually belong to the footprint by using its SpanSet. The DM stack is object oriented, and often classes inherit or contain other classes. That is ture here too. The spanSet is a collection of n spans, where each span has m elements, where n and m are the height and width of the bounding box, respectively.

If an element in a span is zero, this means the pixel is inside the footprint's bounding box but does not belong to the footprint. If an element is one, the pixel is both in the bounding box and associated with the footprint.

Lets dump the output of the spanset from the first footprint in our list

You can almost see the footprint by looking at the 1's and zeros here. To extract the actual pixel values that correspond to the ones in the span, we need an additional step. At the moment, our footprints can tell you if a pixel belongs to it or not, buthave no way of accessing pixel values on the deepCoadd. To remedy this, we will turn our footprint into a HeavyFootprint. HeavyFootprints have all of the qualities of Footprints, but additionally 'know' about pixel level data from the deepCoadd image, variance, and mask planes.

Now we can use the spanset to reassemble the image array into the footprint. Above we saw that the image array is a 1D numpy array-but the footprint itself is 2 dimensional. Fortunately, the span set has an unflatten method that we will use, which can rearrange the image array into the proper 2 dimensional shape

We can do a simmilar exercise with the mask plane values, instead of the image plane values. We can first grab the (flattened) mask plane values of the footprint

To understand these values, lets look at the mask plane's dictionary

The values are the exponent of the bitmask. So Pixels only marked detected will be 2^5 = 32. Pixels that are both on the edge and detected will be 2^5 + 2^4 = 48. Now we will visualize this in a similar manner to the imshow exercise we did before, only now we are only using data for the footprint because we are using the span.

Erosion and Dilation on Footprints¶

We can grow or shrink footprints by using the dilate() and erode() methods. You may imagine doing this if you want to ensure you are getting all the flux associated with the detected source, or break apart one big footprint that contains many sources into several which each contain far fewer. Using these methods requires that we provide a scale by which to grow or shrink the footprint. This scale is often related to the PSF scale, e.g., scaling the RMS or $\sigma$ of a gaussian PSF.

In the dilation process, we can grow out footprint by including its surroundinb pixels. During erosion, we shrinking the object by excluding its outermost pixels.

Let's demonstrate this below

To give us some more intuition on how dilation effects the footprints, we will show cut outs of several footprints before they have been dilated (top row) and after they have been dilated. To exagerate the effect, we will use a large scale length in the dilation. Note the change in the range of the x and y axes in these images.

Footprints, Peaks and Deblending¶

In the previous sections, we paid close attention to a fairly bright, isolated sources and their associated footprints that were identified as being above a 10$\sigma$ threshold. In pratice, especially as we detect with lower thresholds, footprints from a first-pass threshold detection can contain several astronomical objects. This is because the wings of the intensity profiles of individual, but closely packed, sources can bleed into each other, forming one above threshold region, and thus one footprint.

Formally, pixels which are local maxima in the smoothed image during detection are regarded as peaks, and each peak is assumed to be an astrophysical source. In the subsequent deblending process, the deblender looks at all the peaks in each footprints, and gives them their own deblended child footprint. In addition to identifying realestate to deblended children footprints, the deblender also allocates flux from the parent footprint to its children. As a result, the flux in a deblended heavy footprint will not necessarily equal the flux at the same pixels in its image.

In this section we will examine footprints that contain more than one peak, and the interplay between footprints, peaks, and deblending. To do so, we will read footprints from data that has already gone through detection, deblending, and measurement.

In the following few cells, we will use a few afwTable tricks. Because we are interested in understanding peaks and deblending, we are going to sort the source catalog to find the source with the largest number of deblended children. You can learn more about afwTables in the afwTables: A Guided Tour notebook.

The footprint of the parent object contains catalog of the peaks it contains. This catalog gives the peaks and thier locations and peak pixel value in the smoothed image. During the detection process, the image in question is smoothed with a spatial filter. A nice demonstration of this is avaliable in the LowSurfaceBrightness notebook. The peaks' centers and counts are in this smoothed image, and may differ slightly from their subsequent children footprints after deblending.

Now we will set up a visualization to see our parent source, the locations of its peaks (as red crosses), and the centers of the deblended children footprints (as orange circles).

A few comments are in order about the above image.

Even though you can identify by eye there is are several distinct sources here, the detection algorithm grouped them together into one big footprint. Remember, that is totally fine, as footprints can have multiple "peaks" that correspond to different sources. Take a look at one of the large galaxies. It has several peaks associated with it, even though by eye we might call it one large astrophysical object. The deblender shredded these sources into several children sources. This is a known failure mode of deblenders. Deblending is hard, and we will not attempt to solve it here, instead opting to restrict our attention to footprints. If you are interested in learning more about deblending beyond the scope of footprints, Fred Moolekamp has prepared deblending notebooks for the StackClub.

The peaks are marked in the detection process. The peaks are formally detections of distinct astrophysical sources. Notice this is different than Source Extractor. although by eye you can tell they are likely just noise peaks.

The peaks in the parent footprint (red crosses) and the centers of the deblended children are almost all right about on top of each other. There is, however, a region where there is some miss-match. To see this more clearly, we will re-make the plot but zoom in.

Effects of Erosion on Peaks, and Contiguity¶

Now that we have covered erosion/dilation and peaks, we will discuss the interplay between the two. Previously we saw we can change the shape of a footprint by using erosion and dilation operations. Because footprints also contain a list of peaks contained by its span, these operations will in turn effect peaks. Let's continue using the elliptical galaxy with many noise peaks from the previous section.

We will fist plot the elliptical image data and overlay its peaks with red crosses. Then we will erode the eliptical and see the impact this has on peaks.

A few things happened to the footprint after erosion.

1. Its area decreased, as expected

2. The number of peaks decreased. When you use the erode method, any peaks that fall outside the footprint after erosion (called orphan peaks in DM parlance) are discarded. If you happen to alter the span set of a footprint in antother way, you may also use the removeOrphanPeaks method by hand to cull any peaks that are outside the span.

3. The footprint was broken up and is no longer contiguous. We can check the contiguity explicitly

We can remedy this by using the split method on the non-contiguous footprint. This will give us several contiguous footprints, in which all peaks from the initial peak are contained

We have established that deblending can go wrong, and give us unreliable footprints at times. But how does this impact the measurement process? How much flux actually goes to peaks? We will build our intuition for the impact of deblending on heavy footprints in the next section.

Footprints, Noise Replacer, and Deblending¶

As we mentioned at the start of our notebook, footprints work hand in hand with detection, deblending, and measurement. In the following few cells we will loosely follow high level steps that are taken in single frame measurement, so we can understand how footprint deblending can impact measurements and science results.

Refering again to Bosch et al. (2017), the procedure for source measurement is

1. We replace all Footprints in the image with random Gaussian noise with the same variance as the original noise in those pixels.
2. We insert the deblended pixels for a particular source to be measured back into the image (replacing the noise pixels).
3. We run all measurement plug-ins on the current source.
4. We re-replace the Footprint of the current source with noise.
5. Repeat this process for the next Footprint

In our discussion below, we will concentrate on one parent footprint, and visualize its deblended children being swapped back in.

Before we start playing around with swapping footprints in and out of the frame, lets take a look at the calexp as it is. We can see a nice bright source at the center, along with a few distinct fainter sources in the background

Now we will see how we can exchange the detection footprints with noise by using the NoiseReplacer object in the stack. We will emulate steps that are taken in SingleFrameMeasurementTask, where measurement algorithms are run on sources. The next two cells are inspired by a few lines from sfm.py. We will first need to organize our footprints into a dictionary, where the source ID's are the keys and the associated value pairs are the footprints themselves.

We will also point out that you can grab footprints directly from the source catalog.

As we create the noiseReplacer object, noise properties are calculated using the pixel values whose associated mask plane pixels have not been set. This noise distribution is used to replace the footprints with noise that is generated to be representative of the true noise properties of the deepCoadd. Now we will display the deepCoadd again to see the result.

As advertised, we swapped out the data in our footprints with noise. As an aside, if you look closely you can see an 'edge' in the data where the parent object used to be, suggesting perhaps the faint tails of the elliptical's surface brightness profile were missed in detection and now remain in our noise image. We could imagine dilating the footprint to capture these.

Now we will draw some more inspiration from sfm.py where we will insert our parent footprint of interest back into the image using its ID from the source catalog.

We have used our NoiseReplacer instance to ensure this parent footprint is the only source in the image, and all other's have been replaced with noise. The sequence during the measurement process on parent and children footprints would roughly follow like this:

• The noise replacer swaps out all sources with noise
• A parent footprint is restored into the image
• Measurements are run on the parent footprint
• The parent footprint is replaced with noise
• A deblended child footprint is restored into the image
• Measurements are run on the first child

This process would iterate over all parent and child sources. In the next cell we will make a small animation to let us visualize this.

At first glance, in the animation above it seems the central bright galaxies and fainter smaller sources are successfully deblended. However, as the movie progresses we can see that the bright galaxy is shredded by the deblender. This is a common failure mode of the deblender. There are peaks in the outer wings of the galaxy; the deblender receives this list of peaks and assigns flux to them, creating these wispy looking sources that are inserted into the image. Because the deblender conserves the flux of the parent source, any flux allocated to the whispy sources is flux that cannot be allocated to the main child footprint of the main galaxy.

Now we will show a few cut outs of the parent (top left), and first few deblended sources.

It is good pratice to use the end method for the NoiseReplacer after you are finished. This will restore our deepCoadd to the state it was in before we began replacing sources with noise, and so forth

Summary¶

In this notebook, we learned how to make footprints from detection tasks, create heavy footprints, and access pixel level data from heavy footprints. We also learned the role footprints play in detection, deblending, and measurement.

Additional resources:¶

1. Documentation on footprints from the pipelines documentation in the afw package, courtesy Jonathan Sick:
2. Footprint Doxygen page
3. Jim Bosch's HSC Software Pipeline paper, particularly Section 4.8
4. Low Surface Brightness Stack Club Notebook by Alex Drlica-Wagner