imgaug separates its augmentation into two mode: stochastic and deterministic mode. Both perform nearly the same steps. When calling an augmentation function they roughly do:
Hence, if you feed the same batch multiple times into augmenters running in deterministic mode you will get multiple times the exactly same augmentations. Usually, this is not what you want and that is why stochastic mode is the default setting. If however you have two or more batches of images that require the same augmentations, then deterministic mode is the preferred setting. This could be the case e.g. for inputs into siamese networks or for multiple frames from video sequences. The most common use case however are different types data for the same image. E.g. if you have images and bounding boxes on them, the bounding boxes have to be augmented in exactly the same way as the images.
Note: While it is possible to augment multi-input data (e.g. images + bounding boxes) by switching manually to deterministic mode, as described in this notebook, the recommended way is to feed all inputs at the same time into the augmentation call. E.g. one could use
augmenter(images=images, bounding_boxes=bbs) to augment both images and bounding boxes on them in the same way. Aside from
bounding_boxes, the method also accepts
polygons as parameters.
So let's generate an example for the two modes. We will augment two batches of each three identical images. We augment once both batches in stochastic and once both batches in deterministic mode. The result for stochastic mode should differ between the batches, while the one in deterministic mode should be the same for both batches.
import numpy as np import imgaug as ia import imgaug.augmenters as iaa %matplotlib inline ia.seed(2) aug = iaa.Affine(translate_px=(-30, 30), rotate=(-20, 20), cval=255) image = ia.quokka(size=0.15) batches = [[image] * 3, [image] * 3] # two batches of each three images # augment in stochastic mode images_stochastic = [aug(images=batch) for batch in batches] # augment in deterministic mode aug_det = aug.to_deterministic() images_deterministic = [aug_det(images=batch) for batch in batches] # visualize whitespace = np.full(image.shape, 255, dtype=np.uint8) ia.imshow( ia.draw_grid( images_stochastic + [whitespace] + images_stochastic + # first row images_deterministic + [whitespace] + images_deterministic, # second row rows=2, cols=2*3 ) )
The result above shows stochastic mode in the first row (left: batch 1, right: batch 2) and deterministic mode in the second row. As expected, the augmentations for the two batches in stochastic mode differ, but not in deterministic mode. Note though, that even in deterministic mode the results still differ between the images within the same batch.
To provide a more realistic example, we will now augment an image and a set of keypoints on it. The functions for image and keypoint augmentation are in
imgaug separate from each other and because of that we have to switch to deterministic mode before calling them. Otherwise our keypoints would be augmented differently from the images.
Let's load an example image with keypoints on it and visualize the two.
image = ia.quokka(size=0.25) keypoints = ia.quokka_keypoints(size=0.25) ia.imshow( np.hstack([ image, keypoints.draw_on_image(image) ]) )
Now convert the two into batches.
BATCH_SIZE = 4 images_batch = [image] * BATCH_SIZE keypoints_batch = [keypoints] * BATCH_SIZE
Now augment them using affine transformations. We pick a random rotation from a set of possible rotations (multiples of 15). We also use here stochastic mode first and will see further below that this is a bad choice when augmenting images and corresponding keypoints.
aug = iaa.Affine(rotate=[0, 15, 30, 45, 60]) # stochastic mode images_aug1 = aug(images=images_batch) images_aug2 = aug(images=images_batch) images_aug3 = aug(images=images_batch) keypoints_aug1 = aug(keypoints=keypoints_batch) keypoints_aug2 = aug(keypoints=keypoints_batch) keypoints_aug3 = aug(keypoints=keypoints_batch)
And now we visualize the results as a grid. Each row contains the results of one augmented batch.
import matplotlib.pyplot as plt # draw keypoints on images and convert the results to a single image def draw_and_merge(ims, kps): return np.hstack([kp_i.draw_on_image(im_i) for im_i, kp_i in zip(ims, kps)]) fig, axes = plt.subplots(nrows=3, ncols=1, figsize=(15, 8)) axes.imshow(draw_and_merge(images_aug1, keypoints_aug1)) axes.imshow(draw_and_merge(images_aug2, keypoints_aug2)) axes.imshow(draw_and_merge(images_aug3, keypoints_aug3)) for i in range(3): axes[i].set_title("Batch %d" % (i+1,)) axes[i].axis("off")