Notebook

In [ ]:

%matplotlib inline

import openpifpaf
import openpifpaf.logs
openpifpaf.show.Canvas.show = True

import torch
torch.ops.openpifpaf.set_quiet(True)

Cifar10¶

This page gives a quick introduction to OpenPifPaf's Cifar10 plugin that is part of openpifpaf.plugins. It demonstrates the plugin architecture. There already is a nice dataset for CIFAR10 in torchvision and a related PyTorch tutorial. The plugin adds a DataModule that uses this dataset. Let's start with them setup for this notebook and registering all available OpenPifPaf plugins:

In [ ]:

print(openpifpaf.plugin.REGISTERED.keys())

Next, we configure and instantiate the Cifar10 datamodule and look at the configured head metas:

In [ ]:

# configure 
openpifpaf.plugins.cifar10.datamodule.Cifar10.debug = True 
openpifpaf.plugins.cifar10.datamodule.Cifar10.batch_size = 1

# instantiate and inspect
datamodule = openpifpaf.plugins.cifar10.datamodule.Cifar10()
datamodule.set_loader_workers(0)  # no multi-processing to see debug outputs in main thread
datamodule.head_metas

We see here that CIFAR10 is being treated as a detection dataset (CifDet) and has 10 categories. To create a network, we use the factory() function that takes the name of the base network cifar10net and the list of head metas.

In [ ]:

net = openpifpaf.network.Factory(base_name='cifar10net').factory(head_metas=datamodule.head_metas)

We can inspect the training data that is returned from datamodule.train_loader():

In [ ]:

# configure visualization
openpifpaf.visualizer.Base.set_all_indices(['cifdet:9:regression'])  # category 9 = truck

# Create a wrapper for a data loader that iterates over a set of matplotlib axes.
# The only purpose is to set a different matplotlib axis before each call to 
# retrieve the next image from the data_loader so that it produces multiple
# debug images in one canvas side-by-side.
def loop_over_axes(axes, data_loader):
    previous_common_ax = openpifpaf.visualizer.Base.common_ax
    train_loader_iter = iter(data_loader)
    for ax in axes.reshape(-1):
        openpifpaf.visualizer.Base.common_ax = ax
        yield next(train_loader_iter, None)
    openpifpaf.visualizer.Base.common_ax = previous_common_ax

# create a canvas and loop over the first few entries in the training data
with openpifpaf.show.canvas(ncols=6, nrows=3, figsize=(10, 5)) as axs:
    for images, targets, meta in loop_over_axes(axs, datamodule.train_loader()):
        pass

Training¶

We train a very small network, cifar10net, for only one epoch. Afterwards, we will investigate its predictions.

In [ ]:

%%bash
python -m openpifpaf.train \
  --dataset=cifar10 --basenet=cifar10net --log-interval=50 \
  --epochs=3 --lr=0.0003 --momentum=0.95 --batch-size=16 \
  --lr-warm-up-epochs=0.1 --lr-decay 2.0 2.5 --lr-decay-epochs=0.1 \
  --loader-workers=2 --output=cifar10_tutorial.pkl

Plot Training Logs¶

You can create a set of plots from the command line with python -m openpifpaf.logs cifar10_tutorial.pkl.log. You can also overlay multiple runs. Below we call the plotting code from that command directly to show the output in this notebook.

In [ ]:

openpifpaf.logs.Plots(['cifar10_tutorial.pkl.log']).show_all()

Prediction¶

First using CLI:

In [ ]:

%%bash
python -m openpifpaf.predict --checkpoint cifar10_tutorial.pkl.epoch003 images/cifar10_*.png --seed-threshold=0.1 --json-output . --quiet

In [ ]:

%%bash 
cat cifar10_*.json

Using API:

In [ ]:

net_cpu, _ = openpifpaf.network.Factory(checkpoint='cifar10_tutorial.pkl.epoch003').factory()
preprocess = openpifpaf.transforms.Compose([
    openpifpaf.transforms.NormalizeAnnotations(),
    openpifpaf.transforms.CenterPadTight(16),
    openpifpaf.transforms.EVAL_TRANSFORM,
])

openpifpaf.decoder.utils.CifDetSeeds.set_threshold(0.3)
decode = openpifpaf.decoder.factory([hn.meta for hn in net_cpu.head_nets])

data = openpifpaf.datasets.ImageList([
    'images/cifar10_airplane4.png',
    'images/cifar10_automobile10.png',
    'images/cifar10_ship7.png',
    'images/cifar10_truck8.png',
], preprocess=preprocess)
for image, _, meta in data:
    predictions = decode.batch(net_cpu, image.unsqueeze(0))[0]
    print(['{} {:.0%}'.format(pred.category, pred.score) for pred in predictions])

Evaluation¶

I selected the above images, because their category is clear to me. There are images in cifar10 where it is more difficult to tell what the category is and so it is probably also more difficult for a neural network.

Therefore, we should run a proper quantitative evaluation with openpifpaf.eval. It stores its output as a json file, so we print that afterwards.

In [ ]:

%%bash
python -m openpifpaf.eval --checkpoint cifar10_tutorial.pkl.epoch003 --dataset=cifar10 --seed-threshold=0.1 --instance-threshold=0.1 --quiet

In [ ]:

%%bash
python -m json.tool cifar10_tutorial.pkl.epoch003.eval-cifar10.stats.json

We see that some categories like "plane", "car" and "ship" are learned quickly whereas as others are learned poorly (e.g. "bird"). The poor performance is not surprising as we trained our network for a few epochs only.

In [ ]: