很高兴向大家介绍我们的基于 飞桨 开发的可解释性算法开源库 InterpretDL。本文将向大家介绍如何快速上手 InterpretDL。
We introduce InterpretDL, the open source toolkit of interpretation algorithms based on PaddlePaddle. This is a notebook for quickly getting started with InterpretDL.
Before starting coding, we give a quick presentation of InterpretDL. InterpretDL contains a good number of state-of-the-art interpretation algorithms. They can cover a wide range of applications, tasks and models that are required to be interpreted. Meanwhile, InterpretDL also provides several evaluation algorithms to guarantee the trustworthiness of interpretation algorithms. Trustworthiness refers to the loyalty of algorithms to the target model. If an interpretation algorithm gives random explanations that are independent of models, no matter how well the explanations are aligned with human's expectations, that is wrong. So the trustworthiness evaluation algorithms are provided to filter out these disloyal interpretation algorithms.
Below is the content of this tutorial:
First, InterpretDL is based on PaddlePaddle, so PaddlePaddle should be installed in advance. A version with CUDA is recommended.
# install paddlepaddle (cpu). See https://www.paddlepaddle.org.cn/documentation/docs/zh/install/index_cn.html for other installation ways.
%pip install paddlepaddle>=2.2.2 -i https://mirror.baidu.com/pypi/simple
# or paddlepaddle-gpu
# install interpretdl
%pip install interpretdl -i https://mirror.baidu.com/pypi/simple
import paddle
paddle.__version__
'2.2.2'
import interpretdl as it
# version
print('InterpertDL version:', it.__version__)
# available interpretation algorithms
print('InterpretDL algorithms:', it.interpreter.__all__)
InterpertDL version: 0.5.3 InterpretDL algorithms: ['Interpreter', 'InputGradientInterpreter', 'InputOutputInterpreter', 'IntermediateLayerInterpreter', 'LIMECVInterpreter', 'LIMENLPInterpreter', 'GradCAMInterpreter', 'IntGradCVInterpreter', 'IntGradNLPInterpreter', 'SmoothGradInterpreter', 'OcclusionInterpreter', 'GradShapCVInterpreter', 'GradShapNLPInterpreter', 'ScoreCAMInterpreter', 'LRPCVInterpreter', 'RolloutInterpreter', 'TAMInterpreter', 'SmoothGradInterpreterV2', 'ConsensusInterpreter', 'LIMEPriorInterpreter', 'ForgettingEventsInterpreter', 'NormLIMECVInterpreter', 'NormLIMENLPInterpreter']
We have also provided a table, where each implemented interpretation algorithm is categorized by the representation of explanation results and the type of the target model. This taxonomy can be an indicator to find the best suitable algorithm for the target task and model.
Methods | Representation | Model Type |
---|---|---|
LIME | Input Features | Model-Agnostic |
LIME with Prior | Input Features | Model-Agnostic |
NormLIME/FastNormLIME | Input Features | Model-Agnostic |
LRP | Input Features | Differentiable* |
SmoothGrad | Input Features | Differentiable |
IntGrad | Input Features | Differentiable |
GradSHAP | Input Features | Differentiable |
Occlusion | Input Features | Model-Agnostic |
GradCAM/CAM | Intermediate Features | Specific: CNNs |
ScoreCAM | Intermediate Features | Specific: CNNs |
Rollout | Intermediate Features | Specific: Transformers |
TAM | Intermediate Features | Specific: Transformers |
ForgettingEvents | Dataset-Level | Differentiable |
TIDY (Training Data Analyzer) | Dataset-Level | Differentiable |
Consensus | Features | Cross-Model |
* LRP requires that the model is of specific implementations for relevance back-propagation.
There are around 15 algorithms that are available in InterpretDL to provide explanations. Now, suppose we have a trained model, e.g., a ResNet-50 pretrained on ImageNet. Let's see some examples of using interpreters to get explanations of this pretrained ResNet-50.
Load the pretrained ResNet-50 from paddle.vision.
import paddle
device = 'gpu:2'
paddle.set_device(device)
from paddle.vision.models import resnet50
paddle_model = resnet50(pretrained=True)
paddle_model.eval()
# print the structure of ResNet-50
paddle_model
ResNet( (conv1): Conv2D(3, 64, kernel_size=[7, 7], stride=[2, 2], padding=3, data_format=NCHW) (bn1): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (relu): ReLU() (maxpool): MaxPool2D(kernel_size=3, stride=2, padding=1) (layer1): Sequential( (0): BottleneckBlock( (conv1): Conv2D(64, 64, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(64, 64, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(64, 256, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (relu): ReLU() (downsample): Sequential( (0): Conv2D(64, 256, kernel_size=[1, 1], data_format=NCHW) (1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2D(256, 64, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(64, 64, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(64, 256, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (2): BottleneckBlock( (conv1): Conv2D(256, 64, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(64, 64, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=64, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(64, 256, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) ) (layer2): Sequential( (0): BottleneckBlock( (conv1): Conv2D(256, 128, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(128, 128, kernel_size=[3, 3], stride=[2, 2], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(128, 512, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (relu): ReLU() (downsample): Sequential( (0): Conv2D(256, 512, kernel_size=[1, 1], stride=[2, 2], data_format=NCHW) (1): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2D(512, 128, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(128, 128, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(128, 512, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (2): BottleneckBlock( (conv1): Conv2D(512, 128, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(128, 128, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(128, 512, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (3): BottleneckBlock( (conv1): Conv2D(512, 128, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(128, 128, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=128, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(128, 512, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) ) (layer3): Sequential( (0): BottleneckBlock( (conv1): Conv2D(512, 256, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(256, 256, kernel_size=[3, 3], stride=[2, 2], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(256, 1024, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) (relu): ReLU() (downsample): Sequential( (0): Conv2D(512, 1024, kernel_size=[1, 1], stride=[2, 2], data_format=NCHW) (1): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2D(1024, 256, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(256, 256, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(256, 1024, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (2): BottleneckBlock( (conv1): Conv2D(1024, 256, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(256, 256, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(256, 1024, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (3): BottleneckBlock( (conv1): Conv2D(1024, 256, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(256, 256, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(256, 1024, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (4): BottleneckBlock( (conv1): Conv2D(1024, 256, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(256, 256, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(256, 1024, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (5): BottleneckBlock( (conv1): Conv2D(1024, 256, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(256, 256, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=256, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(256, 1024, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=1024, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) ) (layer4): Sequential( (0): BottleneckBlock( (conv1): Conv2D(1024, 512, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(512, 512, kernel_size=[3, 3], stride=[2, 2], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(512, 2048, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=2048, momentum=0.9, epsilon=1e-05) (relu): ReLU() (downsample): Sequential( (0): Conv2D(1024, 2048, kernel_size=[1, 1], stride=[2, 2], data_format=NCHW) (1): BatchNorm2D(num_features=2048, momentum=0.9, epsilon=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2D(2048, 512, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(512, 512, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(512, 2048, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=2048, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) (2): BottleneckBlock( (conv1): Conv2D(2048, 512, kernel_size=[1, 1], data_format=NCHW) (bn1): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (conv2): Conv2D(512, 512, kernel_size=[3, 3], padding=1, data_format=NCHW) (bn2): BatchNorm2D(num_features=512, momentum=0.9, epsilon=1e-05) (conv3): Conv2D(512, 2048, kernel_size=[1, 1], data_format=NCHW) (bn3): BatchNorm2D(num_features=2048, momentum=0.9, epsilon=1e-05) (relu): ReLU() ) ) (avgpool): AdaptiveAvgPool2D(output_size=(1, 1)) (fc): Linear(in_features=2048, out_features=1000, dtype=float32) )
import interpretdl as it
import matplotlib.pyplot as plt
img_path = 'assets/catdog.png'
img, data = it.data_processor.readers.images_transform_pipeline(img_path)
print('img.shape:', img.shape) # img for visualization
print('data.shape:', data.shape) # data for computation.
plt.imshow(img[0])
plt.show()
img.shape: (1, 224, 224, 3) data.shape: (1, 3, 224, 224)
# get human readable labels
!wget -c https://github.com/PaddlePaddle/InterpretDL/files/8561411/readable_label.txt -P ./assets/
f = open('assets/readable_label.txt', 'r')
lines = f.readlines()
# see the predicted top 5 probabilites
probability = paddle.nn.Softmax()(paddle_model(paddle.to_tensor(data)))
topvalues, top_ids = paddle.topk(probability, 5)
print('The model gives the predictions as follows:')
for cls_prob, cls_id in zip(topvalues[0].numpy(), top_ids[0].numpy()):
print(f'\tprobability: {cls_prob:.5f} ({lines[cls_id].strip()})')
The model gives the predictions as follows: probability: 0.32404 (243 bull mastiff) probability: 0.10677 (282 tiger cat) probability: 0.09738 (292 tiger, Panthera tigris) probability: 0.08319 (281 tabby, tabby cat) probability: 0.03909 (285 Egyptian cat)
bull mastiff
and tiger cat
with highest probabilities?¶Let's use InterpretDL to see the explanations.
algo = it.SmoothGradInterpreter(paddle_model, device=device)
explanation = algo.interpret(img_path, labels=243)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:02<00:00, 20.41it/s]
algo = it.LIMECVInterpreter(paddle_model, device=device)
explanation = algo.interpret(img_path, interpret_class=243, num_samples=3000)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3000/3000 [00:20<00:00, 148.74it/s]
interpret
has an argument visual
(defaults to True
), so each time calling interpret
, a visualization is shown. labels
or interpret_class
is the class index to interpret.
Both SmoothGradInterpreter
and LIMECVInterpreter
show the explanations of the prediction 243 (bull mastiff)
, which are aligned with the dog. How about the explanations for 282 (tiger cat)
? We can simply change the class index to interpret.
algo = it.SmoothGradInterpreter(paddle_model, device=device)
explanation = algo.interpret(img_path, labels=282)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:02<00:00, 20.33it/s]
algo = it.LIMECVInterpreter(paddle_model, device=device)
explanation = algo.interpret(img_path, interpret_class=282, num_samples=3000)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3000/3000 [00:20<00:00, 147.36it/s]
We can see that LIMECVInterpreter
gives the explanation on the cat, while SmoothGradInterpreter
on both the dog and the cat. Because SmoothGradInterpreter
and other input-gradient based interpretation algorithms, their visualizations are based on the norm of explanations along the color channel. Some information may be hidden and more analyses should be done.
More Interpreters
are shown here. As we can see, all of them share the same function interpret
while their arguments are slightly different.
GradCAMInterpreter
needs a layer name.
algo = it.GradCAMInterpreter(paddle_model, device=device)
explanation = algo.interpret(img_path, 'layer4.2.conv3', label=243)
explanation = algo.interpret(img_path, 'layer4.2.conv3', label=282)
IntGradCVInterpreter
needs a baselines
.
algo = it.IntGradCVInterpreter(paddle_model, device=device)
explanation = algo.interpret(img_path, baselines='random', labels=243)
explanation = algo.interpret(img_path, baselines='random', labels=282)
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:18<00:00, 27.20it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:18<00:00, 27.35it/s]
We have finished the getting-started tutorial. We have also provided other tutorials, each of them focusing on one specific subject, e.g., deep investigation of algorithms based on input gradients, LIME and its variants, NLP tasks and models, explanations for Transformers, dataset-level interpretations, evaluations of interpretation algorithms' trustworthiness, and so on.
Welcome to contribute and build a better InterpretDL
!