Counterfactual explanations¶

Simple example¶

We start by defining our black-box model, typically represented by

$$ f(\mathbf{x}) = \mathbf{y} $$

Where $\mathbf{x}=\{x_1, x_2, \dots,x_m\}$ and $\mathbf{y}=\{y_1, y_2, \dots,y_n\}$.

Our example toy model, in this case, takes an all-numerical input $\mathbf{x}$ and return a $\mathbf{y}$ of either true or false if the sum of the $\mathbf{x}$ components is within a threshold $\epsilon$ of a point $\mathbf{C}$, that is:

$$ f(\mathbf{x}, \epsilon, \mathbf{C})=\begin{cases} \text{true},\qquad \text{if}\ \mathbf{C}-\epsilon<\sum_{i=1}^m x_i <\mathbf{C}+\epsilon \\ \text{false},\qquad \text{otherwise} \end{cases} $$

This model is provided in the TestUtils module. We instantiate with a $\mathbf{C}=500$ and $\epsilon=1.0$.

Next we need to define a goal. If our model is $f(\mathbf{x'})=\mathbf{y'}$ we are then defining our $\mathbf{y'}$ and the counterfactual result will be the $\mathbf{x'}$ which satisfies $f(\mathbf{x'})=\mathbf{y'}$.

We will define our goal as true, that is, the sum is withing the vicinity of a (to be defined) point $\mathbf{C}$. The goal is a list of Output which take the following parameters

• The feature name
• The feature type
• The feature value (wrapped in Value)
• A confidence threshold, which we will leave at zero (no threshold)

We will now define our initial features, $\mathbf{x}$. Each feature can be instantiated by using FeatureFactory and in this case we want to use numerical features, so we'll use FeatureFactory.newNumericalFeature.

Finally, we also specify which are the bounds for the counterfactual search. Typically this can be set either using domain-specific knowledge or taken from the data. In this case we simply specify an arbitrary (sensible) value, e.g. all the features can vary between 0 and 1000.

As we can see, the sum of of the features will not be within $\epsilon$ (1.0) of $\mathbf{C}$ (500.0). As such the model prediction will be false:

We can now instantiate the explainer itself.

To do so, we will to configure the termination criteria. For this example we will specify that the counterfactual search should only execute a maximum of 10,000 iterations before stopping and returning whatever the best result is so far.

We can can now instantiate the explainer itself using CounterfactualExplainer.

We will now express the counterfactual problem as defined above.

• input represents our $\mathbf{x}$ which know gives a prediction of False
• outputs represents our $\mathbf{y'}$, that is our desired prediction (True)
• domain repreents the boundaries for the counterfactual search

We now request the counterfactual $\mathbf{x'}$ which is closest to $\mathbf{x}$ and which satisfies $f(\mathbf{x'}, \epsilon, \mathbf{C})=\mathbf{y'}$:

We can see that the counterfactual $\mathbf{x'}$

Constrained features¶

As we've seen, it is possible to constraint a specific feature $x_i$ by setting the constraints list corresponding element to True.

In this example, we know want to fix $x_1$ and $x_4$. That is, these features should have the same value in the counterfactual $\mathbf{x'}$ as in the original $\mathbf{x}$.

By simply omitting a search domain to a feature we are constraining it.

And request a new counterfactual explanation

We can see that $x_1$ and $x_4$ has the same value as the original and the model satisfies the conditions.

Using Python models¶

We will now show how to use a custom Python model with TrustyAI counterfactual explanations.

The model will be an XGBoost one trained with the credit-bias dataset (available here).

For convenience, the model is pre-trained and serialised with joblib so that for this example we simply need to deserialised it.

This model has as a single output a boolean PaidLoan, which will predict whether a certain loan applicant will repay the loan in time or not. The model is slightly more complex than the previous examples, with input features:

Input feature	Type	Note
NewCreditCustomer	boolean
Amount	numerical
Interest	numerical
LoanDuration	numerical	In months
Education	numerical	Level (1, 2, 3..)
NrOfDependants	numerical	Integer
EmploymentDurationCurrentEmployer	numerical	Integer (years)
IncomeFromPrincipalEmployer	numerical
IncomeFromPension	numerical
IncomeFromFamilyAllowance	numerical
IncomeFromSocialWelfare	numerical
IncomeFromLeavePay	numerical
IncomeFromChildSupport	numerical
IncomeOther	numerical
ExistingLiabilities	numerical	integer
RefinanceLiabilities	numerical	integer
DebtToIncome	numerical
FreeCash	numerical
CreditScoreEeMini	numerical	integer
NoOfPreviousLoansBeforeLoan	numerical	integer
AmountOfPreviousLoansBeforeLoan	numerical
PreviousRepaymentsBeforeLoan	numerical
PreviousEarlyRepaymentsBefoleLoan	numerical
PreviousEarlyRepaymentsCountBeforeLoan	numerical	integer
Council_house	boolean
Homeless	boolean
Joint_ownership	boolean
Joint_tenant	boolean
Living_with_parents	boolean
Mortgage	boolean
Other	boolean
Owner	boolean
Owner_with_encumbrance	boolean
Tenant	boolean
Entrepreneur	boolean
Fully	boolean
Partially	boolean
Retiree	boolean
Self_employed	boolean

We will start by testing the model with an input we are quite sure (from the original data) that will be predicted as false:

We can confirm now that this input will lead to a false PaidLoan prediction with a probability of $\sim78\%$:

We will now prepare the XGBoost model to be used from the TrustyAI counterfactual engine.

To do so, we simply need to wrap the predict function into a TrustyAI.Modelclass. Since our model outputs a Numpy array, we need to specify the column names of the output, so we can later use those names to set the goals of our counterfactual search.

Unconstrained basic search¶

To get started we will search for a counterfactual with no constraints at all. This is not a realistic use case, but we will use it as a baseline.

We will also create a set of equal bounds for all the features. Again, this is not realistic, but we do it to establish a baseline.

We want our goal to be the model predicting the loan will be paid (PaidLoad=True), so we specify it as:

We are now ready to search for a counterfactual:

First we will confirm that our counterfactual changes the outcome, by predicting its outcome using the model:

And indeed it changes. We will now verify which features were changed:

Here we can see the problem with the unconstrained search.

Some of the fields that were changed might be difficult to change in practice.

Constrained search¶

We will now try a more realistic search, which incorporates domain specific knowledge (and common sence).

To do so, we will constrain features we feel they shouldn't (or mustn't) change and specify sensible search bounds. We will start with the constraints:

The constraints should be self-explanatory, but in essence they were divided into three groups

• Attributes you cannot or should not change (protected), for instance age, education level, etc
• Attributes you can change, for loan duration, loan amount, etc
• Attributes you probably won't be able to change, but might be informative to change. For instance, you might not be able to easily change your income, but you might be interested in how much would it need to be in order to get the prediction as favourable.

And we start a new search:

We test that the counterfactual does change the outcome:

And we confirm that no constrained features were changed:

Minimum counterfactual probabilities¶

We can see that the previous answer is very close to $50\%$.

With TrustyAI we have the possibility to specify a minimum probability for the result (when the model supports prediction confidences).

Let's say we want a result that is at least $75\%$ confident that the loan will be repaid. We can just encode the minimum probability as the last argument of each Output. A minimum probability of $0$ (as we've used) simply means that any desired outcome will be accepted, regardless of its probability.

We can then re-run the search with all the data as defined previously:

As previously, we check that the answer is what we are looking for:

And we show which features need to be changed for said desired outcome:

Visualisation¶

Let's try to visualise counterfactuals in action with the following example:

We construct three clusters of data and train a model to assign points to a cluster. Our counterfactual question can then be

If we have a point $X$ belonging to a certain cluster $C$, how far would it need to move in order to belong to a desired cluster $C^\prime$

We now train a KNN model in order to classify points and assign them to a cluster.

If we take a point such as (2.5, -1) it is clear this point will belong to cluster 2.

We can now create our prediction function. I will simply take the x and y and return the cluster classification, along with a probability.

We wrap this function in the Model wrapper.

We can test it and confirm it works as expected.

We now define our goal. We want it to belong to cluster 1.

We pass this data to the explainer along with the search boundaries.

The counterfactual we get is:

We can confirm this point belongs to cluster 1

And visually: