This notebook is part of the kikuchipy
documentation https://kikuchipy.org.
Links to the documentation won't work from the notebook.
kikuchipy can read and write experimental EBSD patterns and EBSD master patterns from/to multiple formats (see supported formats). To load patterns from file use the load() function. Let's import the necessary libraries and read the Nickel EBSD test data set directly from file (not via kikuchipy.data.nickel_ebsd_small()):
# exchange inline for qt5 for interactive plotting from the pyqt package
%matplotlib inline
import tempfile
import dask.array as da
import hyperspy.api as hs
import kikuchipy as kp
import numpy as np
import matplotlib.pyplot as plt
datadir = "../kikuchipy/data/"
nordif_ebsd = "nordif/Pattern.dat"
s = kp.load(datadir + nordif_ebsd)
s
Or, load the stereographic projection of the northern hemisphere of an EBSD master
pattern for a 20 keV beam energy from a modified version of EMsoft's master
pattern file, returned from their EMEBSDmaster.f90
program:
emsoft_master_pattern = (
"emsoft_ebsd_master_pattern/ni_mc_mp_20kv_uint8_gzip_opts9.h5"
)
s_mp = kp.load(filename=datadir + emsoft_master_pattern)
s_mp
Both the stereographic and the square Lambert projections of this master pattern data is available via kikuchipy.data.nickel_ebsd_master_pattern_small().
All file readers support accessing the data without loading it into memory (with the Dask library), which can be useful when processing large data sets to avoid memory errors:
s_lazy = kp.load(datadir + nordif_ebsd, lazy=True)
print(s_lazy)
s_lazy.data
Parts or all of the data can be read into memory by calling compute():
s_lazy_copy = s_lazy.inav[:2, :].deepcopy()
s_lazy_copy.compute()
s_lazy_copy
s_lazy.compute()
s_lazy
Note
When lazily loaded EBSD patterns are processed, they are processed chunk by chunk, which in many cases leads to longer processing times, so processing lazy data sets should be done with some care. See the relevant HyperSpy user guide for information on how to do this.
Visualization of data is done by navigating navigation space, showing the signal in each navigation point:
s.plot()
Upon loading, kikuchipy tries to read all scan information from the file and
stores everything it can read in the original_metadata
attribute:
# s.original_metadata # Long output
Also, some information may be stored in a standard location in the metadata
attribute where it can be used by EBSD class methods:
s.metadata
The number of patterns in horizontal and vertical direction, pattern size in
pixels, scan step size and detector pixel size is stored in the axes_manager
attribute:
s.axes_manager
This information can be modified directly, and information in metadata
and
axes_manager
can also be modified by the
EBSD class methods
set_experimental_parameters(),
set_phase_parameters(),
set_scan_calibration() and
set_detector_calibration().
For example, to set or change the accelerating voltage, horizontal pattern
centre coordinate and static background pattern (stored as a numpy.ndarray
):
s.set_experimental_parameters(
beam_energy=15,
xpc=0.5073,
static_background=plt.imread(
datadir + "nordif/Background acquisition pattern.bmp"
)
)
In addition to the HyperSpy provided metadata
, original_metadata
and
axes_manager
properties, kikuchipy tries to read a CrystalMap object with indexing results into a
xmap
property and an
EBSDDetector
object into a detector
property:
s.xmap # This is empty unless it is set
s.detector
An EBSD
or EBSDMasterPattern
signal can also be created directly from a
numpy.ndarray
. To create a data set of (60 x 60) pixel patterns in a
(10 x 20) grid, i.e. 10 and 20 patterns in the horizontal and vertical scan
directions respectively, of random intensities:
s_np = kp.signals.EBSD(np.random.random((20, 10, 60, 60)))
s_np
When processing large data sets, it is useful to load data lazily with the
Dask library. This can be done upon reading patterns from a file
by setting lazy=True
when using the load()
function, or directly from a
dask.array.Array
:
s_da = kp.signals.LazyEBSD(
da.random.random((20, 10, 60, 60), chunks=(2, 10, 60, 60))
)
print(s_da)
s_da.data
HyperSpy provides the method
set_signal_type()
to change between BaseSignal subclasses, of which
EBSD
, EBSDMasterPattern
and
VirtualBSEImage are three. To
get one of these objects from a HyperSpy Signal2D
object:
s_hs = hs.signals.Signal2D(np.random.random((20, 10, 60, 60)))
s_hs
s_hs.set_signal_type("EBSD")
s_hs
s_hs.set_signal_type("VirtualBSEImage")
s_hs
s_hs.set_signal_type("EBSDMasterPattern")
s_hs
To save experimental EBSD patterns to file use the
save() method. For example, to save
an EBSD
signal in an HDF5 file, with file name patterns.h5
, in our default
h5ebsd format:
temp_dir = tempfile.mkdtemp()
s.save(temp_dir + "patterns")
Warning
If we want to overwrite an existing file:
s.save("patterns.h5", overwrite=True)
If we want to save patterns in NORDIF's binary .dat format instead:
s.save(temp_dir + "patterns.dat")
To save an EBSDMasterPattern
to an HDF5 file, we use the save method inherited from HyperSpy
to write to their HDF5 specification:
s_hs.save(temp_dir + "master_pattern.hspy")
s_hs
These master patterns can then be read into an EBSDMasterPattern
signal again
via HyperSpy's
load():
s_mp2 = hs.load(
temp_dir + "master_pattern.hspy", signal_type="EBSDMasterPattern"
)
s_mp2
Note
To save results from statistical decomposition (machine learning) of patterns to
file see the section
Saving and loading results
in HyperSpy's user guide. Note that the file extension .hspy
must be used upon
saving, s.save('patterns.hspy')
, as the default extension in kikuchipy, .h5
,
yields a kikuchipy h5ebsd file where the decomposition results aren't saved. The
saved patterns can then be reloaded using HyperSpy's load()
function and
passing the signal_type="EBSD"
parameter
as explained above.
Currently, kikuchipy has readers and writers for the following formats:
Format | Read | Write |
---|---|---|
Bruker Nano h5ebsd | Yes | No |
EDAX TSL h5ebsd | Yes | No |
kikuchipy h5ebsd | Yes | Yes |
NORDIF binary | Yes | Yes |
EMsoft simulated EBSD HDF5 | Yes | No |
EMsoft EBSD master pattern HDF5 | Yes | No |
Bruker Nano h5ebsd | Yes | No |
Note
If you want to process your patterns with kikuchipy, but use an unsupported EBSD vendor software, or if you want to write your processed patterns to a vendor format that does not support writing, please request this feature in our issue tracker.
The h5ebsd format Jackson et al. (2014) is based on the HDF5 open standard (Hierarchical Data Format version 5). HDF5 files can be read and edited using e.g. the HDF Group's reader HDFView or the Python package used by kikuchipy, h5py. Upon loading an HDF5 file with extension .h5, .hdf5, or .h5ebsd, the correct reader is determined from the file. Supported h5ebsd formats are listed in the table above.
If an h5ebsd file contains multiple scans, as many scans as desirable can be
read from the file. For example, if the file contains two scans with names
My awes0m4 Xcan #! with a long title
and Scan 2
:
kikuchipy_ebsd = "kikuchipy/patterns.h5"
s_awsm, s2 = kp.load(
filename=datadir + kikuchipy_ebsd,
scan_group_names=["My awes0m4 Xcan #! with a long title", "Scan 2"]
)
print(s_awsm)
print(s2)
Here, the h5ebsd
file_reader() is
called. If only Scan 2
is to be read, scan_group_names="Scan 2"
can be
passed:
s2 = kp.load(filename=datadir + kikuchipy_ebsd, scan_group_names="Scan 2")
s2
The scan_group_names
parameter is unnecessary if only the first scan in the
file is to be read, since reading only the first scan in the file is the default
behaviour.
So far, only saving patterns to kikuchipy's own h5ebsd format
is supported. It is possible to write a new scan with a scan name Scan x
,
where x
is an integer, to an existing, but closed, h5ebsd file in the
kikuchipy format, e.g. one containing only Scan 1
, by passing:
new_file = "patterns_new.h5"
s2.save(temp_dir + new_file, scan_number=1)
s_awsm.save(filename=temp_dir + new_file, add_scan=True, scan_number=2)
s2_new, s_awsm_new = kp.load(
filename=temp_dir + new_file, scan_group_names=["Scan 1", "Scan 2"]
)
print(s2_new)
print(s_awsm_new)
Here, the h5ebsd file_writer() is called.
Note
The EBSD.xmap
and EBSD.detector
properties are so far not written to this
file format.
Patterns acquired using NORDIF's acquisition software are stored in a binary
file usually named Pattern.dat
. Scan information is stored in a separate text
file usually named Setting.txt
, and both files usually reside in the same
directory. If this is the case, the patterns can be loaded by passing the file
name as the only parameter. If this is not the case, the setting file can be
passed upon loading:
s_nordif = kp.load(
filename=datadir + nordif_ebsd, setting_file=datadir + "nordif/Setting.txt"
)
s_nordif
Here, the NORDIF file_reader() is called. If the scan information, i.e. scan and pattern size, in the setting file is incorrect or the setting file is not available, patterns can be loaded by passing:
s_nordif = kp.load(
filename=datadir + nordif_ebsd, scan_size=(1, 9), pattern_size=(60, 60)
)
s_nordif
If a static background pattern named Background acquisition.bmp
is stored in
the same directory as the pattern file, this is stored in metadata
upon
loading.
Patterns can also be saved to a NORDIF binary file, upon which the NORDIF file_writer() is called. Note, however, that so far no new setting file, background pattern, or calibration patterns are created upon saving.
Dynamically simulated EBSD patterns returned by EMsoft's EMEBSD.f90
program
as HDF5 files can be read as an EBSD
signal:
emsoft_ebsd = "emsoft_ebsd/simulated_ebsd.h5" # Dummy data set
s_sim = kp.load(filename=datadir + emsoft_ebsd)
s_sim
Here, the EMsoft simulated EBSD
file_reader() is
called, which takes the optional argument scan_size
. Passing
scan_size=(2, 5)
will reshape the pattern data shape from (10, 10, 10)
to
(2, 5, 10, 10)
:
s_sim2 = kp.load(filename=datadir + emsoft_ebsd, scan_size=(2, 5))
print(s_sim2)
print(s_sim2.data.shape)
Simulated EBSD patterns can be written to the kikuchipy h5ebsd format, the NORDIF binary format, or to HDF5 files using HyperSpy's HDF5 specification as explained above.
Master patterns returned by EMsoft's EMEBSDmaster.f90
program as HDF5 files
can be read as an EBSDMasterPattern
signal:
s_mp = kp.load(filename=datadir + emsoft_master_pattern)
print(s_mp)
print(s_mp.projection)
print(s_mp.hemisphere)
print(s_mp.phase)
Here, the EMsoft EBSD master pattern
file_reader()
is called, which takes the optional arguments projection
, hemisphere
and
energy
. The stereographic projection is read by default. Passing
projection="lambert"
will read the square Lambert projection instead. The
northern hemisphere is read by default. Passing hemisphere="south"
or
hemisphere="both"
will read the southern hemisphere projection or both,
respectively. Master patterns for all beam energies are read by default. Passing
energy=(10, 20)
or energy=15
will read the master pattern(s) with beam
energies from 10 to 20 keV, or just 15 keV, respectively:
s_mp = kp.load(
datadir + emsoft_master_pattern,
projection="lambert",
hemisphere="both",
energy=20
)
print(s_mp)
print(s_mp.projection)
print(s_mp.hemisphere)
Master patterns can be written to HDF5 files using HyperSpy's HDF5 specification as explained above.
See Jackson et al. (2019) for a hands-on tutorial explaining how to simulate these patterns with EMsoft, and Callahan & De Graef (2013) for details of the underlying theory.
Patterns saved in the h5ebsd format can be read by the
dictionary indexing and related routines in
EMsoft using the EMEBSD
reader. Those
routines in EMsoft also have a NORDIF
reader.
Patterns saved in the h5ebsd format can of course be read in Python like any other HDF5 data set:
import h5py
with h5py.File(datadir + kikuchipy_ebsd, mode="r") as f:
dset = f['Scan 2/EBSD/Data/patterns']
print(dset)
patterns = dset[()]
print(patterns.shape)
plt.figure()
plt.imshow(patterns[0], cmap="gray")
plt.axis("off")
One or more virtual backscatter electron (BSE) images in a VirtualBSEImage signal can be read and written to file using one of HyperSpy's many readers and writers. If they are only to be used internally in HyperSpy, they can be written to and read back from HyperSpy's HDF5 specification as explained above for EBSD master patterns.
If we want to write the images to image files, HyperSpy also provides a series of image readers/writers, as explained in their IO user guide. If we wanted to write them as a stack of TIFF images:
# Get virtual image from generator
vbse_gen = kp.generators.VirtualBSEGenerator(s)
print(vbse_gen)
print(vbse_gen.grid_shape)
vbse = vbse_gen.get_images_from_grid()
print(vbse)
vbse.rescale_intensity()
vbse.unfold_navigation_space() # 1D navigation space required for TIFF
vbse
vbse_fname = "vbse.tif"
vbse.save(temp_dir + vbse_fname) # Easily read into e.g. ImageJ
We can also write them to e.g. png
or bmp
files with Matplotlib
:
nav_size = vbse.axes_manager.navigation_size
_ = [
plt.imsave(temp_dir + f"vbse{i}.png", vbse.inav[i].data)
for i in range(nav_size)
]
Read the TIFF stack back into a VirtualBSEImage
signal:
vbse2 = hs.load(temp_dir + vbse_fname, signal_type="VirtualBSEImage")
vbse2
import os
os.rmdir(temp_dir)