Quickstart¶

In this notebook, we will explore visual, a companion package to torchbearer that enables complex feature visualisation with minimal work.

We shall first show how easy it is to get a feature visualisation going using visual.

In [1]:

!pip install -q torchbearer-visual

We import a couple things from visual and then print the layer names of squeezenet v1.1 so that we can decide which layer to ascend on.

In [2]:

from visual.models import inception_v3
from visual import BasicAscent, Channel, image
from visual.transforms import RandomRotate, RandomScale, SpatialJitter, Compose
from visual.models.utils import IntermediateLayerGetter

return_layers = {'Mixed_5b': 'feat1'}
model = IntermediateLayerGetter(inception_v3(True, False), return_layers)
print(model.layer_names)

['Conv2d_1a_3x3', 'Conv2d_1a_3x3_conv', 'Conv2d_1a_3x3_bn', 'Conv2d_1a_3x3_relu', 'Conv2d_2a_3x3', 'Conv2d_2a_3x3_conv', 'Conv2d_2a_3x3_bn', 'Conv2d_2a_3x3_relu', 'Conv2d_2b_3x3', 'Conv2d_2b_3x3_conv', 'Conv2d_2b_3x3_bn', 'Conv2d_2b_3x3_relu', 'MaxPool2b', 'Conv2d_3b_1x1', 'Conv2d_3b_1x1_conv', 'Conv2d_3b_1x1_bn', 'Conv2d_3b_1x1_relu', 'Conv2d_4a_3x3', 'Conv2d_4a_3x3_conv', 'Conv2d_4a_3x3_bn', 'Conv2d_4a_3x3_relu', 'MaxPool4a', 'Mixed_5b', 'Mixed_5b_branch1x1', 'Mixed_5b_branch1x1_conv', 'Mixed_5b_branch1x1_bn', 'Mixed_5b_branch1x1_relu', 'Mixed_5b_branch5x5_1', 'Mixed_5b_branch5x5_1_conv', 'Mixed_5b_branch5x5_1_bn', 'Mixed_5b_branch5x5_1_relu', 'Mixed_5b_branch5x5_2', 'Mixed_5b_branch5x5_2_conv', 'Mixed_5b_branch5x5_2_bn', 'Mixed_5b_branch5x5_2_relu', 'Mixed_5b_branch3x3dbl_1', 'Mixed_5b_branch3x3dbl_1_conv', 'Mixed_5b_branch3x3dbl_1_bn', 'Mixed_5b_branch3x3dbl_1_relu', 'Mixed_5b_branch3x3dbl_2', 'Mixed_5b_branch3x3dbl_2_conv', 'Mixed_5b_branch3x3dbl_2_bn', 'Mixed_5b_branch3x3dbl_2_relu', 'Mixed_5b_branch3x3dbl_3', 'Mixed_5b_branch3x3dbl_3_conv', 'Mixed_5b_branch3x3dbl_3_bn', 'Mixed_5b_branch3x3dbl_3_relu', 'Mixed_5b_AvgPool', 'Mixed_5b_branch_pool', 'Mixed_5b_branch_pool_conv', 'Mixed_5b_branch_pool_bn', 'Mixed_5b_branch_pool_relu', 'Mixed_5c', 'Mixed_5c_branch1x1', 'Mixed_5c_branch1x1_conv', 'Mixed_5c_branch1x1_bn', 'Mixed_5c_branch1x1_relu', 'Mixed_5c_branch5x5_1', 'Mixed_5c_branch5x5_1_conv', 'Mixed_5c_branch5x5_1_bn', 'Mixed_5c_branch5x5_1_relu', 'Mixed_5c_branch5x5_2', 'Mixed_5c_branch5x5_2_conv', 'Mixed_5c_branch5x5_2_bn', 'Mixed_5c_branch5x5_2_relu', 'Mixed_5c_branch3x3dbl_1', 'Mixed_5c_branch3x3dbl_1_conv', 'Mixed_5c_branch3x3dbl_1_bn', 'Mixed_5c_branch3x3dbl_1_relu', 'Mixed_5c_branch3x3dbl_2', 'Mixed_5c_branch3x3dbl_2_conv', 'Mixed_5c_branch3x3dbl_2_bn', 'Mixed_5c_branch3x3dbl_2_relu', 'Mixed_5c_branch3x3dbl_3', 'Mixed_5c_branch3x3dbl_3_conv', 'Mixed_5c_branch3x3dbl_3_bn', 'Mixed_5c_branch3x3dbl_3_relu', 'Mixed_5c_AvgPool', 'Mixed_5c_branch_pool', 'Mixed_5c_branch_pool_conv', 'Mixed_5c_branch_pool_bn', 'Mixed_5c_branch_pool_relu', 'Mixed_5d', 'Mixed_5d_branch1x1', 'Mixed_5d_branch1x1_conv', 'Mixed_5d_branch1x1_bn', 'Mixed_5d_branch1x1_relu', 'Mixed_5d_branch5x5_1', 'Mixed_5d_branch5x5_1_conv', 'Mixed_5d_branch5x5_1_bn', 'Mixed_5d_branch5x5_1_relu', 'Mixed_5d_branch5x5_2', 'Mixed_5d_branch5x5_2_conv', 'Mixed_5d_branch5x5_2_bn', 'Mixed_5d_branch5x5_2_relu', 'Mixed_5d_branch3x3dbl_1', 'Mixed_5d_branch3x3dbl_1_conv', 'Mixed_5d_branch3x3dbl_1_bn', 'Mixed_5d_branch3x3dbl_1_relu', 'Mixed_5d_branch3x3dbl_2', 'Mixed_5d_branch3x3dbl_2_conv', 'Mixed_5d_branch3x3dbl_2_bn', 'Mixed_5d_branch3x3dbl_2_relu', 'Mixed_5d_branch3x3dbl_3', 'Mixed_5d_branch3x3dbl_3_conv', 'Mixed_5d_branch3x3dbl_3_bn', 'Mixed_5d_branch3x3dbl_3_relu', 'Mixed_5d_AvgPool', 'Mixed_5d_branch_pool', 'Mixed_5d_branch_pool_conv', 'Mixed_5d_branch_pool_bn', 'Mixed_5d_branch_pool_relu', 'Mixed_6a', 'Mixed_6a_branch3x3', 'Mixed_6a_branch3x3_conv', 'Mixed_6a_branch3x3_bn', 'Mixed_6a_branch3x3_relu', 'Mixed_6a_branch3x3dbl_1', 'Mixed_6a_branch3x3dbl_1_conv', 'Mixed_6a_branch3x3dbl_1_bn', 'Mixed_6a_branch3x3dbl_1_relu', 'Mixed_6a_branch3x3dbl_2', 'Mixed_6a_branch3x3dbl_2_conv', 'Mixed_6a_branch3x3dbl_2_bn', 'Mixed_6a_branch3x3dbl_2_relu', 'Mixed_6a_branch3x3dbl_3', 'Mixed_6a_branch3x3dbl_3_conv', 'Mixed_6a_branch3x3dbl_3_bn', 'Mixed_6a_branch3x3dbl_3_relu', 'Mixed_6a_MaxPool', 'Mixed_6b', 'Mixed_6b_branch1x1', 'Mixed_6b_branch1x1_conv', 'Mixed_6b_branch1x1_bn', 'Mixed_6b_branch1x1_relu', 'Mixed_6b_branch7x7_1', 'Mixed_6b_branch7x7_1_conv', 'Mixed_6b_branch7x7_1_bn', 'Mixed_6b_branch7x7_1_relu', 'Mixed_6b_branch7x7_2', 'Mixed_6b_branch7x7_2_conv', 'Mixed_6b_branch7x7_2_bn', 'Mixed_6b_branch7x7_2_relu', 'Mixed_6b_branch7x7_3', 'Mixed_6b_branch7x7_3_conv', 'Mixed_6b_branch7x7_3_bn', 'Mixed_6b_branch7x7_3_relu', 'Mixed_6b_branch7x7dbl_1', 'Mixed_6b_branch7x7dbl_1_conv', 'Mixed_6b_branch7x7dbl_1_bn', 'Mixed_6b_branch7x7dbl_1_relu', 'Mixed_6b_branch7x7dbl_2', 'Mixed_6b_branch7x7dbl_2_conv', 'Mixed_6b_branch7x7dbl_2_bn', 'Mixed_6b_branch7x7dbl_2_relu', 'Mixed_6b_branch7x7dbl_3', 'Mixed_6b_branch7x7dbl_3_conv', 'Mixed_6b_branch7x7dbl_3_bn', 'Mixed_6b_branch7x7dbl_3_relu', 'Mixed_6b_branch7x7dbl_4', 'Mixed_6b_branch7x7dbl_4_conv', 'Mixed_6b_branch7x7dbl_4_bn', 'Mixed_6b_branch7x7dbl_4_relu', 'Mixed_6b_branch7x7dbl_5', 'Mixed_6b_branch7x7dbl_5_conv', 'Mixed_6b_branch7x7dbl_5_bn', 'Mixed_6b_branch7x7dbl_5_relu', 'Mixed_6b_AvgPool', 'Mixed_6b_branch_pool', 'Mixed_6b_branch_pool_conv', 'Mixed_6b_branch_pool_bn', 'Mixed_6b_branch_pool_relu', 'Mixed_6c', 'Mixed_6c_branch1x1', 'Mixed_6c_branch1x1_conv', 'Mixed_6c_branch1x1_bn', 'Mixed_6c_branch1x1_relu', 'Mixed_6c_branch7x7_1', 'Mixed_6c_branch7x7_1_conv', 'Mixed_6c_branch7x7_1_bn', 'Mixed_6c_branch7x7_1_relu', 'Mixed_6c_branch7x7_2', 'Mixed_6c_branch7x7_2_conv', 'Mixed_6c_branch7x7_2_bn', 'Mixed_6c_branch7x7_2_relu', 'Mixed_6c_branch7x7_3', 'Mixed_6c_branch7x7_3_conv', 'Mixed_6c_branch7x7_3_bn', 'Mixed_6c_branch7x7_3_relu', 'Mixed_6c_branch7x7dbl_1', 'Mixed_6c_branch7x7dbl_1_conv', 'Mixed_6c_branch7x7dbl_1_bn', 'Mixed_6c_branch7x7dbl_1_relu', 'Mixed_6c_branch7x7dbl_2', 'Mixed_6c_branch7x7dbl_2_conv', 'Mixed_6c_branch7x7dbl_2_bn', 'Mixed_6c_branch7x7dbl_2_relu', 'Mixed_6c_branch7x7dbl_3', 'Mixed_6c_branch7x7dbl_3_conv', 'Mixed_6c_branch7x7dbl_3_bn', 'Mixed_6c_branch7x7dbl_3_relu', 'Mixed_6c_branch7x7dbl_4', 'Mixed_6c_branch7x7dbl_4_conv', 'Mixed_6c_branch7x7dbl_4_bn', 'Mixed_6c_branch7x7dbl_4_relu', 'Mixed_6c_branch7x7dbl_5', 'Mixed_6c_branch7x7dbl_5_conv', 'Mixed_6c_branch7x7dbl_5_bn', 'Mixed_6c_branch7x7dbl_5_relu', 'Mixed_6c_AvgPool', 'Mixed_6c_branch_pool', 'Mixed_6c_branch_pool_conv', 'Mixed_6c_branch_pool_bn', 'Mixed_6c_branch_pool_relu', 'Mixed_6d', 'Mixed_6d_branch1x1', 'Mixed_6d_branch1x1_conv', 'Mixed_6d_branch1x1_bn', 'Mixed_6d_branch1x1_relu', 'Mixed_6d_branch7x7_1', 'Mixed_6d_branch7x7_1_conv', 'Mixed_6d_branch7x7_1_bn', 'Mixed_6d_branch7x7_1_relu', 'Mixed_6d_branch7x7_2', 'Mixed_6d_branch7x7_2_conv', 'Mixed_6d_branch7x7_2_bn', 'Mixed_6d_branch7x7_2_relu', 'Mixed_6d_branch7x7_3', 'Mixed_6d_branch7x7_3_conv', 'Mixed_6d_branch7x7_3_bn', 'Mixed_6d_branch7x7_3_relu', 'Mixed_6d_branch7x7dbl_1', 'Mixed_6d_branch7x7dbl_1_conv', 'Mixed_6d_branch7x7dbl_1_bn', 'Mixed_6d_branch7x7dbl_1_relu', 'Mixed_6d_branch7x7dbl_2', 'Mixed_6d_branch7x7dbl_2_conv', 'Mixed_6d_branch7x7dbl_2_bn', 'Mixed_6d_branch7x7dbl_2_relu', 'Mixed_6d_branch7x7dbl_3', 'Mixed_6d_branch7x7dbl_3_conv', 'Mixed_6d_branch7x7dbl_3_bn', 'Mixed_6d_branch7x7dbl_3_relu', 'Mixed_6d_branch7x7dbl_4', 'Mixed_6d_branch7x7dbl_4_conv', 'Mixed_6d_branch7x7dbl_4_bn', 'Mixed_6d_branch7x7dbl_4_relu', 'Mixed_6d_branch7x7dbl_5', 'Mixed_6d_branch7x7dbl_5_conv', 'Mixed_6d_branch7x7dbl_5_bn', 'Mixed_6d_branch7x7dbl_5_relu', 'Mixed_6d_AvgPool', 'Mixed_6d_branch_pool', 'Mixed_6d_branch_pool_conv', 'Mixed_6d_branch_pool_bn', 'Mixed_6d_branch_pool_relu', 'Mixed_6e', 'Mixed_6e_branch1x1', 'Mixed_6e_branch1x1_conv', 'Mixed_6e_branch1x1_bn', 'Mixed_6e_branch1x1_relu', 'Mixed_6e_branch7x7_1', 'Mixed_6e_branch7x7_1_conv', 'Mixed_6e_branch7x7_1_bn', 'Mixed_6e_branch7x7_1_relu', 'Mixed_6e_branch7x7_2', 'Mixed_6e_branch7x7_2_conv', 'Mixed_6e_branch7x7_2_bn', 'Mixed_6e_branch7x7_2_relu', 'Mixed_6e_branch7x7_3', 'Mixed_6e_branch7x7_3_conv', 'Mixed_6e_branch7x7_3_bn', 'Mixed_6e_branch7x7_3_relu', 'Mixed_6e_branch7x7dbl_1', 'Mixed_6e_branch7x7dbl_1_conv', 'Mixed_6e_branch7x7dbl_1_bn', 'Mixed_6e_branch7x7dbl_1_relu', 'Mixed_6e_branch7x7dbl_2', 'Mixed_6e_branch7x7dbl_2_conv', 'Mixed_6e_branch7x7dbl_2_bn', 'Mixed_6e_branch7x7dbl_2_relu', 'Mixed_6e_branch7x7dbl_3', 'Mixed_6e_branch7x7dbl_3_conv', 'Mixed_6e_branch7x7dbl_3_bn', 'Mixed_6e_branch7x7dbl_3_relu', 'Mixed_6e_branch7x7dbl_4', 'Mixed_6e_branch7x7dbl_4_conv', 'Mixed_6e_branch7x7dbl_4_bn', 'Mixed_6e_branch7x7dbl_4_relu', 'Mixed_6e_branch7x7dbl_5', 'Mixed_6e_branch7x7dbl_5_conv', 'Mixed_6e_branch7x7dbl_5_bn', 'Mixed_6e_branch7x7dbl_5_relu', 'Mixed_6e_AvgPool', 'Mixed_6e_branch_pool', 'Mixed_6e_branch_pool_conv', 'Mixed_6e_branch_pool_bn', 'Mixed_6e_branch_pool_relu', 'AuxLogits', 'AuxLogits_AvgPool0', 'AuxLogits_conv0', 'AuxLogits_conv0_conv', 'AuxLogits_conv0_bn', 'AuxLogits_conv0_relu', 'AuxLogits_conv1', 'AuxLogits_conv1_conv', 'AuxLogits_conv1_bn', 'AuxLogits_conv1_relu', 'AuxLogits_AvgPool1', 'AuxLogits_fc', 'Mixed_7a', 'Mixed_7a_branch3x3_1', 'Mixed_7a_branch3x3_1_conv', 'Mixed_7a_branch3x3_1_bn', 'Mixed_7a_branch3x3_1_relu', 'Mixed_7a_branch3x3_2', 'Mixed_7a_branch3x3_2_conv', 'Mixed_7a_branch3x3_2_bn', 'Mixed_7a_branch3x3_2_relu', 'Mixed_7a_branch7x7x3_1', 'Mixed_7a_branch7x7x3_1_conv', 'Mixed_7a_branch7x7x3_1_bn', 'Mixed_7a_branch7x7x3_1_relu', 'Mixed_7a_branch7x7x3_2', 'Mixed_7a_branch7x7x3_2_conv', 'Mixed_7a_branch7x7x3_2_bn', 'Mixed_7a_branch7x7x3_2_relu', 'Mixed_7a_branch7x7x3_3', 'Mixed_7a_branch7x7x3_3_conv', 'Mixed_7a_branch7x7x3_3_bn', 'Mixed_7a_branch7x7x3_3_relu', 'Mixed_7a_branch7x7x3_4', 'Mixed_7a_branch7x7x3_4_conv', 'Mixed_7a_branch7x7x3_4_bn', 'Mixed_7a_branch7x7x3_4_relu', 'Mixed_7a_MaxPool', 'Mixed_7b', 'Mixed_7b_branch1x1', 'Mixed_7b_branch1x1_conv', 'Mixed_7b_branch1x1_bn', 'Mixed_7b_branch1x1_relu', 'Mixed_7b_branch3x3_1', 'Mixed_7b_branch3x3_1_conv', 'Mixed_7b_branch3x3_1_bn', 'Mixed_7b_branch3x3_1_relu', 'Mixed_7b_branch3x3_2a', 'Mixed_7b_branch3x3_2a_conv', 'Mixed_7b_branch3x3_2a_bn', 'Mixed_7b_branch3x3_2a_relu', 'Mixed_7b_branch3x3_2b', 'Mixed_7b_branch3x3_2b_conv', 'Mixed_7b_branch3x3_2b_bn', 'Mixed_7b_branch3x3_2b_relu', 'Mixed_7b_branch3x3dbl_1', 'Mixed_7b_branch3x3dbl_1_conv', 'Mixed_7b_branch3x3dbl_1_bn', 'Mixed_7b_branch3x3dbl_1_relu', 'Mixed_7b_branch3x3dbl_2', 'Mixed_7b_branch3x3dbl_2_conv', 'Mixed_7b_branch3x3dbl_2_bn', 'Mixed_7b_branch3x3dbl_2_relu', 'Mixed_7b_branch3x3dbl_3a', 'Mixed_7b_branch3x3dbl_3a_conv', 'Mixed_7b_branch3x3dbl_3a_bn', 'Mixed_7b_branch3x3dbl_3a_relu', 'Mixed_7b_branch3x3dbl_3b', 'Mixed_7b_branch3x3dbl_3b_conv', 'Mixed_7b_branch3x3dbl_3b_bn', 'Mixed_7b_branch3x3dbl_3b_relu', 'Mixed_7b_AvgPool', 'Mixed_7b_branch_pool', 'Mixed_7b_branch_pool_conv', 'Mixed_7b_branch_pool_bn', 'Mixed_7b_branch_pool_relu', 'Mixed_7c', 'Mixed_7c_branch1x1', 'Mixed_7c_branch1x1_conv', 'Mixed_7c_branch1x1_bn', 'Mixed_7c_branch1x1_relu', 'Mixed_7c_branch3x3_1', 'Mixed_7c_branch3x3_1_conv', 'Mixed_7c_branch3x3_1_bn', 'Mixed_7c_branch3x3_1_relu', 'Mixed_7c_branch3x3_2a', 'Mixed_7c_branch3x3_2a_conv', 'Mixed_7c_branch3x3_2a_bn', 'Mixed_7c_branch3x3_2a_relu', 'Mixed_7c_branch3x3_2b', 'Mixed_7c_branch3x3_2b_conv', 'Mixed_7c_branch3x3_2b_bn', 'Mixed_7c_branch3x3_2b_relu', 'Mixed_7c_branch3x3dbl_1', 'Mixed_7c_branch3x3dbl_1_conv', 'Mixed_7c_branch3x3dbl_1_bn', 'Mixed_7c_branch3x3dbl_1_relu', 'Mixed_7c_branch3x3dbl_2', 'Mixed_7c_branch3x3dbl_2_conv', 'Mixed_7c_branch3x3dbl_2_bn', 'Mixed_7c_branch3x3dbl_2_relu', 'Mixed_7c_branch3x3dbl_3a', 'Mixed_7c_branch3x3dbl_3a_conv', 'Mixed_7c_branch3x3dbl_3a_bn', 'Mixed_7c_branch3x3dbl_3a_relu', 'Mixed_7c_branch3x3dbl_3b', 'Mixed_7c_branch3x3dbl_3b_conv', 'Mixed_7c_branch3x3dbl_3b_bn', 'Mixed_7c_branch3x3dbl_3b_relu', 'Mixed_7c_AvgPool', 'Mixed_7c_branch_pool', 'Mixed_7c_branch_pool_conv', 'Mixed_7c_branch_pool_bn', 'Mixed_7c_branch_pool_relu', 'AvgPool', 'fc']

Let's choose the final layer, where we have 1000 channels, each corresponding to 1 imagenet class. We'll choose the 256th feature, which is the class "Newfoundland Dog". After running a BasicAscent and viewing it in pyplot, you can confirm for yourself that we indeed see things that look like dogs.

In [5]:

transforms = Compose([
    RandomRotate(list(range(-30, 30, 5))),
    RandomScale([0.9, 0.95, 1.0, 1.05, 1.1, 1.15, 1.2]),
])

crit = Channel(10, 'feat1')
img = image((3, 256, 256), transform=transforms, correlate=True, fft=True)


a = BasicAscent(img, crit, verbose=2).to_pyplot().run(model, device='cuda')

HBox(children=(IntProgress(value=0, description='0/1(t)', max=256, style=ProgressStyle(description_width='init…

So, we have seen how quick it is to get a simple visualisation running, but what were did we actually do to get there?

We'll now introduce, one at a time, the abstractions that visual uses:

Images
Transforms
Criterions
Ascenders
Models

In [ ]: