Handle changes in prediction during robustness evaluation.¶

Typically, during robustness evaluation, we want model prediction to stay the same. This behaviour is, however, very sensitive to perturbation function and its hyperparameter choices. In this notebook we demonstrate how this could be handled in quantus using a simple motivating example with Average Sensitivity Metric.

In [12]:

# Import dependencies.
import pandas as pd
import tensorflow as tf
import tensorflow_datasets as tfds
import quantus

tf.config.list_physical_devices()

Out[12]:

[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]

1) Preliminaries¶

1.1 Load ImageNet subset.¶

In [13]:

IMG_SIZE = 224

ds = tfds.load(
    "imagenet_v2",
    split=["test"],
    as_supervised=True,
    try_gcs=True,
    batch_size=32,
    data_dir="/tmp/tensorflow_datasets/",
)

x_batch, y_batch = ds[0].take(1).as_numpy_iterator().next()
x_batch = tf.image.resize(x_batch, (IMG_SIZE, IMG_SIZE)).numpy()
x_batch.shape

Out[13]:

(32, 224, 224, 3)

1.2. Load pre-trained model.¶

In [14]:

model = tf.keras.applications.MobileNetV2()
model.input

Out[14]:

<KerasTensor: shape=(None, 224, 224, 3) dtype=float32 (created by layer 'input_3')>

1.3. Generate batch of predictions and explanations using baseline method "IntegratedGradients".¶

In [15]:

y_predicted = model.predict(x_batch).argmax(axis=1)
a_batch_intgrad = quantus.explain(
    model, x_batch, y_predicted, method="IntegratedGradients"
)
a_batch_intgrad.shape

Out[15]:

(32, 224, 224)

2) Quantitative evaluation with Quantus¶

We can evaluate the robustness of our explanations on a variety of quantitative criteria, but as a motivating example we test the Average Sensitivity (Yeh at el., 2019) of the explanations. This metric tests how the explanations change on average while subject to slight perturbations.

All robustness metrics accept constructor keyword argument return_nan_when_prediction_changes, as the name suggests, when set to true, the metric will be evaluated to NaN if the prediction changes after the perturbation is applied.

In [16]:

results = quantus.evaluate(
    metrics={
        "DefaultAvgSensitivity": quantus.AvgSensitivity(
            nr_samples=20, disable_warnings=True, display_progressbar=True
        ),
        "AvgSensitivityWithNan": quantus.AvgSensitivity(
            nr_samples=20,
            disable_warnings=True,
            return_nan_when_prediction_changes=True,
            display_progressbar=True,
        ),
    },
    xai_methods={"IntegratedGradients": a_batch_intgrad},
    model=model,
    x_batch=x_batch,
    y_batch=y_batch,
    explain_func_kwargs={"method": "IntegratedGradients"},
    call_kwargs={"0": {"softmax": True, "channel_first": True}},
)

  0%|          | 0/1 [00:00<?, ?it/s]

2.2 Display results in tabular form.¶

In [17]:

pd.DataFrame(
    [
        results["IntegratedGradients"]["DefaultAvgSensitivity"],
        results["IntegratedGradients"]["AvgSensitivityWithNan"],
    ],
    index=["No Prediction Change Check", "Nan On Prediction Change"],
)

Out[17]:

	0	1	2	3	4	5	6	7	8	9	...	22	23	24	25	26	27	28	29	30	31
No Prediction Change Check	0.003274	0.003258	0.008044	0.005117	0.007698	0.004486	0.003926	0.00226	0.004641	0.009831	...	0.005673	0.0055	0.004242	0.007323	0.005993	0.007763	0.009405	0.012347	0.004398	0.011894
Nan On Prediction Change	NaN	NaN	0.008044	0.005117	0.007699	0.004486	NaN	NaN	NaN	0.009832	...	0.005673	0.0055	NaN	0.007323	0.005996	0.007763	0.009405	0.012347	NaN	NaN

2 rows × 32 columns