This notebook is part of the kikuchipy documentation https://kikuchipy.org. Links to the documentation won't work from the notebook.

Hybrid indexing¶

In this tutorial we will combine Hough indexing (HI), dictionary indexing (DI) and refinement in a hybrid indexing approach. Hough indexing is generally much faster than dictionary indexing, but less robust towards noisy patterns. To make good use of both indexing approaches, we can quickly index all patterns with Hough indexing, identify badly indexed map points and re-index these with dictionary indexing. Before combining the Hough indexed and dictionary indexed points into a single map in the final step, we must refine them. As always, it is important to evaluate intermediate results after each step by inspecting quality metrics, geometrical simulations etc.

We will demonstrate this workflow on a single phase recrystallized nickel sample. The dataset is available from Zenodo at Ånes et al. (2019). It is dataset number 10 (24 dB) out of the series of 10 datasets taken with increasing gain (0-24 dB). It is very noisy, so we will average each pattern to its nearest neighbours before indexing.

The complete workflow then becomes:

1. Load, process and inspect the full dataset
2. Calibrate geometry by getting a plane of projection centers
3. Hough indexing of all patterns
4. Identify points for re-indexing
5. Re-indexing with dictionary indexing
6. Refine Hough indexed and dictionary indexed points
7. Merge results
8. Validate results

Let's start by important the necessary libraries

Load, process and inspect data¶

Enhance signal-to-noise ratio (background correction)

Average each pattern to its eight nearest neighbors in a Gaussian kernel with a standard deviation of 1

Inspect an image quality map (pattern sharpness, not to be confused with image quality determined from height of peaks in the Hough transform)

Inspect patterns in the image quality map

The image quality map is very noisy, but we might be able to convince ourselves that the darker lines are grain boundaries (they are). The map is noisy because the patterns are noisy. The pattern shown is from inside one of the larger grains on the right side of the map from the small red square (the pointer). Even though the material is well recrystallized with appreciably large grains, the pattern is very noisy. But again, we might be able to convince ourselves that there is some "correlated noise", e.g. a couple of zones axes (darker regions in the pattern) and some bands delineated by darker lines on each side of the band...

Calibrate geometry¶

Seven calibration patterns of high quality was acquired prior to acquiring the full dataset above, in order to calibrate the sample-detector geometry. The detector was mounted with 0$^{\circ}$, and we will assume that the nominal sample tilt of 70$^{\circ}$ is correct. What remains then is to determine a plane of projection centers (PCs), one for each point in the full dataset. Since we know the detector pixel size (~70 $\mu$m on the NORDIF UF-1100 detector), we can extrapolate this plane of PCs from a mean PC. The workflow is the following:

1. Estimate PCs from an initial guess using Hough indexing
2. Hough indexing of calibration patterns using estimated PCs
3. Refine Hough indexed orientations and estimated PCs using pattern matching
4. Extrapolate plane of PCs from mean of refined PCs

After each step, we will evaluate our results.

Load calibration patterns

Remove static and dynamic background

Extract positions of calibration patterns on the sample surface (possibly outside the region of interest [ROI]) and the shape and position of the ROI relative to the area imaged in an overview secondary electron image. This information is read with the calibration patterns from the NORDIF settings file.

Plot calibration pattern map locations

Hough indexing requires a phase list in order to make a look-up table of interplanar angles to compare the detected angles (from combinations of bands) to. So far, Hough indexing only supports face-centered and body-centered cubic (FCC and BCC) materials (dictionary indexing and pattern matching support all crystallographic point groups). See the hough indexing tutorial for more details. Since we later on need a dynamically simulated master pattern of nickel (simulated with EMsoft), we will load this here and use the phase attached to this master pattern in the phase list.

Create a PyEBSDIndex indexer to use with Hough indexing. We get this from our EBSD detector instance attached to the calibration pattern signal

Estimate PCs from an initial guess based on previous experiments, and print the mean and standard deviation

Compare the distribution of PCs to the above plotted map locations (especially PCx vs. PCy)

We see no direct correlation between the sample positions and the PCs. Let's index the calibration patterns using these PCs and compare the solution's band positions to the experimental band positions. We update our indexer instance with the estimated PCs.

Check indexed orientations by plotting geometrical simulations on top of the patterns. See the tutorial on geometrical EBSD simulations for details.

Most lines align quite well with experimental bands, but some of them, especially in the lower parts of the patterns and on wide bands, do not follow the band middle. Let's refine these solutions using pattern matching

Check quality metrics

Check deviations from Hough indexed solutions (it is important that these deviations are well within our trust region above)

Get geometrical simulations from refined orientations and PCs and add lines from these simulations (in red) to the existing figure

We see that the red lines align better with wider bands and bands in the lower part of the patterns (where the deviations are also greater).

Check the refined PCs

We now see that pattern 0, 3, 4 and 6 seem to align quite well compared to the sample positions. However, the other PCs do not align as expected. It is safest in this instance to compute a mean PC and extrapolate from this.

Get sample positions of our patterns

Specify the outliers

Extrapolate a plane of PCs

Check the plane of PCs

As a final evaluation of this plane of PCs, we will refine the (refined) orientations using a fixed PC for each pattern taken from the plane of PCs

Get geometrical simulations and add a third set of lines (in blue) to the existing figure

Hough indexing of all patterns¶

Now that we are confident of our geometry calibration, we can index all patterns in our noisy experimental dataset.

Copy the detector with the calibrated PCs and update the detector shape to match our experimental patterns

Get a new indexer

Perform Hough indexing with PyEBSDIndex (using the GPU via PyOpenCL, but only a single CPU)

Our use of PyEBSDIndex here gave indexing of about 900 patterns/s.

PyEBSDIndex couldn't index some 200 points (pattern fit above 3$^{\circ}$). Let's check the quality metrics (pattern fit, confidence metric, pattern quality and the number of detected bands labeled)

The bright points in the lower right pattern fit map are the 213 points considered not indexed. The confidence metric and number of successfully labeled bands (out of nine) seem to be highest within grains and lowest at grain boundaries. Let's inspect the spatial variation of "successfully" indexed orientations in an inverse pole figure map (IPF-X)

Many points seen as single color deviations from otherwise smooth colors within recrystallized grains are located mostly at grain boundaries.

Identify points for re-indexing¶

Let's see if we can easily separate the good from bad points using any of the quality metrics

... hm, there is no clear bimodal distribution in any of the histograms. It is recommended to find the "good" and "bad" points by trial-and-error until a desired separation is achieved. Here, we have combined metrics by trial-and-error to get a plausible separation, but note that other combinations might be better for other datasets.

There are some spurious points still left among the points to keep, otherwise the orientation map looks fairly convincing.

Make navigation mask 2D. Points to re-index are set to False.

Re-indexing with dictionary indexing¶

To generate the dictionary of Ni patterns, we need to sample the orientation space at a sufficiently high resolution (here 2$^{\circ}$) with a fixed calibration geometry (PC). See the pattern matching tutorial for details.

We only match the intensities within a circular mask (note the inversion!)

Perform dictionary indexing of the patterns and intensities marked as False in the navigation and signal masks

We see that HI is about 6-7x faster than DI.

An average correlation score of 0.15 is low, but OK since we can refine the solutions and would expect a higher score from this. The orientation map looks plausible, though, which it did not from these patterns after HI.

Refine Hough indexed and dictionary indexed points¶

First we specify common refinement parameters, so that the scores obtain can be compared. This is very important!

Of the Hough indexed solutions, we only want to refine those that are not re-indexed using dictionary indexing. We therefore pass the navigation mask, but have to take care to set those points that we want to index to False

An average correlation score of 0.25 is OK. We now refine the re-indexed points. No navigation mask is necessary, since the crystal map returned from DI has a mask keeping track of which points are "in the data" via CrystalMap.is_in_data.

An average correlation score of 0.20 is still OK. We see that refinement of these points needed slightly more function evaluations, presumably because they were harder to index due to them being mostly located at grain boundaries and having less clear bands.

Merge results¶

We can now merge the results, taking care to pass the navigation mask where each refined map should be considered. Since only the points not in the refined HI map were indexed with DI, the same mask can be used in both cases.

We see that we have a complete map for all our points!

Evaluate results¶

Finally, we can compare the IPF-X maps with HI only and after re-indexing, refinement and combination

We extract a grid of patterns and plot the geometrical simulations on top of these patterns

It's difficult to see any bands in these very noisy patterns... There at least seems to be a correlation between darker regions in the patterns (not the corners) and zone axes, which is expected.

In conclusion, by combining the speed of Hough indexing (about 6-7x faster than DI) with the robustness towards noise of dictionary indexing, a dataset can be indexed in a shorter time and achieve about the same results as with DI (and refinement) only.