Notebook

Automating Abstraction Error¶

After exploring the idea of abstraction [Rischel2020] and transformation [Rubenstein2017] in the first three notebooks, we now focus on the more practical problem of how, given two SCMs and an abstraction between them, we can compute the abstraction error between them.

In this notebook we will develop code to compute exactly the abstraction error as defined in [Rischel2020] and in the previous notebook. To achieve our goal, we will write functions that will be dependant on the metric we want to adopt and on the set of nodes over which we will evaluate the abstraction error. We will present a series of implementation progressively considering richer sets over which to compute the abstraction error.

This notebook was developed in order to offer a practical implementation of the ideas presented in the framework introduced in [Rischel2020], and to lay stonger foundations to further work with the idea of abstraction of causal models. The notebook is structured as follows:

Setup of standard and custom libraries (Section 2)
Review of the definition of abstraction and notation (Section 3)
Automated computation of abstraction error given pairs of variables in M1 connected by a directed path in M1 (Section 4)
Automated computation of abstraction error given all pairs of variables in M1 (Section 5)
Automated computation of abstraction error given pairs of variables in M1 connected by a directed path in M0 or M1 (Section 6)
Automated computation of abstraction error given sets of variables in M1 connected by a directed path in M0 or M1 (Section 7)
Integration of the code for the automated computation in the Abstraction class (Section 8)

DISCLAIMER 1: the notebook refers to ideas from causality and category theory for which only a quick definition is offered. Useful references for causality are [Pearl2009,Peters2017], while for category theory are [Spivak2014,Fong2018].

DISCLAIMER 2: mistakes are in all likelihood due to misunderstandings by the notebook author in reading [Rischel2020]. Feedback very welcome! :)

Setup¶

Standard libraries¶

First of all we import standard libraries.

In [1]:

import numpy as np
import networkx as nx
import itertools
from scipy.spatial import distance

For reproducibility, and for discussing our results in this notebook, we set a random seed to $1985$.

In [2]:

np.random.seed(1985)

SCM implemenation¶

So far we have used a custom class to define and run our SCMs; we defined this class in the previous notebooks and we exported it into src/models.py. We will now switch to a standard library to implement Bayesian networks (BN): pgmpy. pgmpy will allow us to define our SCMs as BNs and use existing functions to perform inferences on our network.

To use pgmpy we import the relevant libraries:

In [3]:

from pgmpy.models import BayesianNetwork as BN
from pgmpy.factors.discrete import TabularCPD as cpd
from pgmpy.inference import VariableElimination

Abstraction definition¶

To be compatible with our new objects, we need to slightly revise the Abstraction class. Before, our class expected to work with custom FinStochSCM objects; now it will work with BayesianNetwork objects.

Importantly, we had to redefine the components of an abstraction:

R is now a list of strings, each one denoting a variable in the base model $\mathcal{M}$/$\mathtt{M0}$ deemed relevant;
a is now a dictionary indexed by the relevant variables in R; it specifies on which variable in the abstracted model $\mathcal{M'}$/$\mathtt{M1}$ the relevant variables are mapped.
alphas is now dictionary indexed by variables in the abstracted model $\mathcal{M'}$/$\mathtt{M1}$; it specifies a matrix defining how variable(s) in the base model $\mathcal{M}$/$\mathtt{M0}$ are mapped onto a variable in the abstracted model $\mathcal{M'}$/$\mathtt{M1}$.

A tweaked implementation of the Abstraction class able to deal with these inputs has been written in src/abstraction.py. We import it.

In [4]:

from src.legacy.SCMMappings_1_0 import Abstraction

Abstraction error refresher¶

Before moving to implementing a module for the automatic computation of error, let us review the definition of the abstraction error $e(\alpha)$.

Let's $\mathbf{A}$ and $\mathbf{B}$ two sets of variables in the base model $\mathcal{M}$/$\mathtt{M0}$; let's $X$ and $Y$ be the related variables in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$. Let us now consider the following diagram:

$$ \begin{array}{ccc} \mathcal{\mathcal{M}_{do}}\left[\mathbf{A}\right] & \overset{\mathcal{\mathcal{M}_{do}}\left[\phi_{\mathbf{B}}\right]}{\longrightarrow} & \mathcal{\mathcal{M}_{do}}\left[\mathbf{B}\right]\\ \sideset{}{\alpha_{X}}\downarrow & & \sideset{}{\alpha_{Y}}\downarrow\\ \mathcal{\mathcal{M'}_{do}}\left[X\right] & \overset{\mathcal{\mathcal{M'}_{do}}\left[\phi_{Y}\right]}{\longrightarrow} & \mathcal{\mathcal{M'}_{do}}\left[Y\right] \end{array} $$

The abstraction error $E_\alpha(X,Y)$ with respect to the variables $X,Y$ is the JS distance under intervention between the upper path and the lower path:

$$ D_{JS}(\alpha_Y \circ \mathcal{M}_{do}\left[\phi_{\mathbf{B}}\right], \mathcal{M'}_{do}\left[\phi_{Y}\right] \circ \alpha_X) $$

The overall abstraction error $e(\alpha)$ is the highest abstraction error $E_\alpha(X,Y)$ considering all the meaningful pairs of variables $(X,Y)$ in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$. We call the set of pair of variables $(X,Y)$ over which the error is estimated the evaluation set $\mathcal{J}$.

A note on the convention¶

To facilitate the understanding of the code and clarify how input models and abstractions should be defined, it is useful to write a short note on the representation convention for the matrices.

All morphisms we are dealing with (abstraction maps $\alpha_\cdot$ and mechanisms $\mathcal{M}[\phi_\cdot]$) are defined as stochastic matrices.

Abstractions. An abstraction $\alpha_X$ maps a set of variables $\mathbf{A}$ in the base model $\mathcal{M}$/$\mathtt{M0}$ to a single variable $X$ in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$. Let $\mathcal{M}[A_1] \times \mathcal{M}[A_2] \times ... \times \mathcal{M}[A_n]$ be the domain of $\mathbf{A}$, and $\mathcal{M'}[X]$ the domain of $X$. Then, the matrix encoding $\alpha_X$ will be a two-dimensional matrix with dimensions:

$$ \left[\left|\mathcal{M'}[X]\right|, \left|\mathcal{M}[A_1]\right|\cdot\left|\mathcal{M}[A_2]\right|\cdot...\cdot\left|\mathcal{M}[A_n]\right|\right] $$

For instance, consider $\alpha_{X}:A_1\times A_2\rightarrow X$ and suppose:

$\mathcal{M}[A_1]=\{0,1,2\}$,
$\mathcal{M}[A_2]=\{0,1\}$,
$\mathcal{M'}[X]=\{0,1,2,3\}$

then, $\alpha_{X}$ is a stochastic matrix with dimensions $[4,6]$ and the following interpretation:

$$ \begin{array}{ccccccc} & \boldsymbol{(A_{1}=0,A_{2}=0)} & \boldsymbol{(A_{1}=0,A_{2}=1)} & \boldsymbol{(A_{1}=1,A_{2}=0)} & \boldsymbol{(A_{1}=1,A_{2}=1)} & \boldsymbol{(A_{1}=2,A_{2}=0)} & \boldsymbol{(A_{1}=2,A_{2}=1)}\\ \boldsymbol{(X=0)} & (0,0)\mapsto0 & (0,1)\mapsto0 & (1,0)\mapsto0 & (1,1)\mapsto0 & (2,0)\mapsto0 & (2,1)\mapsto0\\ \boldsymbol{(X=1)} & (0,0)\mapsto1 & (0,1)\mapsto1 & (1,0)\mapsto1 & (1,1)\mapsto1 & (2,0)\mapsto1 & (2,1)\mapsto1\\ \boldsymbol{(X=2)} & (0,0)\mapsto2 & (0,1)\mapsto2 & (1,0)\mapsto2 & (1,1)\mapsto2 & (2,0)\mapsto2 & (2,1)\mapsto2\\ \boldsymbol{(X=3)} & (0,0)\mapsto3 & (0,1)\mapsto3 & (1,0)\mapsto3 & (1,1)\mapsto3 & (2,0)\mapsto3 & (2,1)\mapsto3 \end{array} $$

Mechanisms. A mechanism $\mathcal{M}[\phi_B]$ maps a set of variables $\mathbf{A}$ to a a set of variables $\mathbf{B}$ in the same model according to the conditional $P(\mathbf{B} \vert \mathbf{A})$. Let $\mathcal{M}[A_1] \times \mathcal{M}[A_2] \times ... \times \mathcal{M}[A_n]$ be the domain of $\mathbf{A}$, and $\mathcal{M}[B_1] \times \mathcal{M}[B_2] \times ... \times \mathcal{M}[B_m]$ the domain of $\mathbf{B}$. Then, the matrix encoding $\mathcal{M}[\phi_B]$ will be a two-dimensional matrix with dimensions:

$$ \left[\left|\mathcal{M}[B_1]\right|\cdot\left|\mathcal{M}[B_2]\right|\cdot...\cdot\left|\mathcal{M}[B_m]\right|, \left|\mathcal{M}[A_1]\right|\cdot\left|\mathcal{M}[A_2]\right|\cdot...\cdot\left|\mathcal{M}[A_n]\right|\right] $$

For instance, consider $\mathcal{M}[\phi_B]:A_1\times A_2\rightarrow B_1\times B_2$ and suppose:

$\mathcal{M}[A_1]=\{0,1,2\}$,
$\mathcal{M}[A_2]=\{0,1\}$,
$\mathcal{M}[B_1]=\{0,1\}$,
$\mathcal{M}[B_2]=\{0,1,2,3\}$,

then, $\mathcal{M}[\phi_B]$ is a stochastic matrix with dimensions $[8,6]$ and the following interpretation:

$$ \begin{array}{ccccc} & \boldsymbol{(A_{1}=0,A_{2}=0)} & \boldsymbol{(A_{1}=0,A_{2}=1)} & & \boldsymbol{(A_{1}=2,A_{2}=1)}\\ \boldsymbol{(B_{1}=0,B_{2}=0)} & P(B_{1}=0,B_{2}=0\vert A_{1}=0,A_{2}=0) & P(B_{1}=0,B_{2}=0\vert A_{1}=0,A_{2}=1) & & P(B_{1}=0,B_{2}=0\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=0,B_{2}=1)} & P(B_{1}=0,B_{2}=1\vert A_{1}=0,A_{2}=0) & P(B_{1}=0,B_{2}=1\vert A_{1}=0,A_{2}=1) & & P(B_{1}=0,B_{2}=1\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=0,B_{2}=2)} & P(B_{1}=0,B_{2}=2\vert A_{1}=0,A_{2}=0) & P(B_{1}=0,B_{2}=2\vert A_{1}=0,A_{2}=1) & ... & P(B_{1}=0,B_{2}=2\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=0,B_{2}=3)} & P(B_{1}=0,B_{2}=3\vert A_{1}=0,A_{2}=0) & P(B_{1}=0,B_{2}=3\vert A_{1}=0,A_{2}=1) & & P(B_{1}=0,B_{2}=3\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=1,B_{2}=0)} & P(B_{1}=1,B_{2}=0\vert A_{1}=0,A_{2}=0) & P(B_{1}=1,B_{2}=0\vert A_{1}=0,A_{2}=1) & & P(B_{1}=1,B_{2}=0\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=1,B_{2}=1)} & P(B_{1}=1,B_{2}=1\vert A_{1}=0,A_{2}=0) & P(B_{1}=1,B_{2}=1\vert A_{1}=0,A_{2}=1) & & P(B_{1}=1,B_{2}=1\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=1,B_{2}=2)} & P(B_{1}=1,B_{2}=2\vert A_{1}=0,A_{2}=0) & P(B_{1}=1,B_{2}=2\vert A_{1}=0,A_{2}=1) & & P(B_{1}=1,B_{2}=2\vert A_{1}=2,A_{2}=1)\\ \boldsymbol{(B_{1}=1,B_{2}=3)} & P(B_{1}=1,B_{2}=3\vert A_{1}=0,A_{2}=0) & P(B_{1}=1,B_{2}=3\vert A_{1}=0,A_{2}=1) & & P(B_{1}=1,B_{2}=3\vert A_{1}=2,A_{2}=1) \end{array} $$

Composition. This encoding (codomain on the row, domain on the column; product of multiple finite sets in row-order) is consistent with the encoding used in pgmpy and it allows us to perform matrix multiplications in the same order of categorical composition.

Evaluating abstraction considering only individual variables connected by a directed path¶

We start implementing our algorithm for the automatic computation of the abstraction error using as evaluation set $\mathcal{J}$ the pairs $(X,Y)$ such that there exist a directed path from $X$ to $Y$ in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$.

Example 1¶

To drive our implementation, we work using as a reference the prototypical example we have studied in the previous notebooks. We consider models for the lung cancer scenario at different levels. A precise description of each one of them is available in the notebook Categorical Abstraction.ipynb.

Model definition¶

We first define our two models. Notice how our models are now defined using pgmpy; we also take advantage of the check_model() function offered by pgmpy to verify that our model and its stochastic matrices are correct.

In [5]:

M0 = BN([('Smoking','Tar'),('Tar','Cancer')])

cpdS = cpd(variable='Smoking',
          variable_card=2,
          values=[[.8],[.2]],
          evidence=None,
          evidence_card=None)
cpdT = cpd(variable='Tar',
          variable_card=2,
          values=[[1,.2],[0.,.8]],
          evidence=['Smoking'],
          evidence_card=[2])
cpdC = cpd(variable='Cancer',
          variable_card=2,
          values=[[.9,.6],[.1,.4]],
          evidence=['Tar'],
          evidence_card=[2])

M0.add_cpds(cpdS,cpdT,cpdC)
M0.check_model()

Out[5]:

True

In [6]:

M1 = BN([('Smoking','Cancer')])

cpdS = cpd(variable='Smoking',
          variable_card=2,
          values=[[.8],[.2]],
          evidence=None,
          evidence_card=None)
cpdC = cpd(variable='Cancer',
          variable_card=2,
          values=[[.9,.66],[.1,.34]],
          evidence=['Smoking'],
          evidence_card=[2])

M1.add_cpds(cpdS,cpdC)
M1.check_model()

Out[6]:

True

Next we define an abstraction. The definition of $(R,a,\alpha)$ has correspondingly changed to work with the pgmpy objects.

In [7]:

R = ['Smoking','Cancer']

a = {'Smoking': 'Smoking',
    'Cancer': 'Cancer'}
alphas = {'Smoking': np.eye(2),
         'Cancer': np.eye(2)}

In [8]:

A = Abstraction(M0,M1,R,a,alphas)

Enumeration of all possible pairs in M1 with a directed path connection in M1¶

We now populate our evaluation set $\mathcal{J}$ computing all possible pairs (source,target) along directed paths in the abstracted model $\mathcal{M'}/\mathtt{M1}$. To do so, we call the networkx function has_path() among all possible pairs of nodes.

In [9]:

J = []

sources = list(A.M1.nodes())
targets = list(A.M1.nodes())

for s in sources:
    for t in list(set(targets)-{s}):
        if nx.has_path(A.M1,s,t):
            J.append((s,t))

In [10]:

print(J)

[('Smoking', 'Cancer')]

As obviosly clear, we have a single path in the abstracted model.

Running the computational loop¶

We now come to the loop that will compute the abstraction error $e(\alpha)$. The loop iterates over all the pairs in the evaluation set $\mathcal{J}$ and computing the specific abstraction error.

Keeping in mind this diagram:

let us decompose and explain the loop:

Given a pair from the evaluation set $\mathcal{J}$, extract the source node $X$ and the target node $Y$ in the abstracted model $\mathcal{M'}/\mathtt{M1}$.
Using the mapping $a$, retrieve the corresponding source set $\mathbf{A}$ and target set $\mathbf{B}$ in the base model $\mathcal{M}/\mathtt{M0}$.
Compute the intervention $do(X)$ in the abstracted model $\mathcal{M'}/\mathtt{M1}$ and initialize the pgmpy inference engine on this intervened model.
Compute the intervention $do(\mathbf{A})$ in the base model $\mathcal{M}/\mathtt{M0}$ and initialize the pgmpy inference engine on this intervened model.
Compute the mechanism $\mathcal{M'}_{do}[\phi_Y] = P_{{M'}_{do}}(Y\vert X)$ as $\frac{P_{{M'}_{do}}(Y,X)}{P_{{M'}_{do}}(X)}$. The two distributions $P_{{M'}_{do}}(Y,X)$ and $P_{{M'}_{do}}(X)$ are evaluated using the pgmpy inference engine.
Extract the matrix form of $\mathcal{M'}_{do}[\phi_Y] = P_{{M'}_{do}}(Y\vert X)$.
Check if the matrix correctly represents $Y$ on the rows and $X$ on the columns.
Compute the mechanism $\mathcal{M}_{do}[\phi_\mathbf{B}] = P_{{M}_{do}}(\mathbf{B}\vert \mathbf{A})$ as $\frac{P_{{M}_{do}}(\mathbf{B},\mathbf{A})}{P_{{M}_{do}}(\mathbf{A})}$. The two distributions $P_{{M}_{do}}(\mathbf{B},\mathbf{A})$ and $P_{{M}_{do}}(\mathbf{A})$ are evaluated using the pgmpy inference engine.
Extract the matrix form of $\mathcal{M}_{do}[\phi_\mathbf{B}] = P_{{M'}_{do}}(\mathbf{B}\vert \mathbf{A})$.
Reorder the matrix to be sure that the variables in $\mathbf{B}$ are coming before the variables in $\mathbf{A}$, and that they are in order they are supposed to be.
Compact the matrix reducing it to two dimensions: on the rows we have the variables of $\mathbf{B}$, on the columns the varibles of $\mathbf{A}$.
Extract the matrices for the abstractions of interest: $\alpha_X$ and $\alpha_Y$.
Compute the two alternative path on the diagram by composing $\mathcal{M'}_{do}[\phi_Y] \circ \alpha_X$ (lower path) and $\alpha_Y \circ \mathcal{M}_{do}[\phi_\mathbf{B}]$ (upper path) via a simple matrix product.
For every possible intervention on $\mathbf{A}$ compute the JS distance between the two paths. This is equivalent to consider every column in the matrix encoding of the two paths and computing their JS distance.
Select the highest distance with respect to all the interventions as the error $E_\alpha(Y,X)$.
Select the highest distance with repsect to all pairs $(X,Y)$ as the error $e(\alpha)$.

In [11]:

abstraction_errors = []

for pair in J:
    # Get nodes in the abstracted model
    M1_source = [pair[0]]
    M1_target = [pair[1]]
    print('\nM1: {0} -> {1}'.format(M1_source,M1_target))

    # Get nodes in the base model
    M0_source = A.invert_a(M1_source)
    M0_target = A.invert_a(M1_target)
    print('M0: {0} -> {1}'.format(M0_source,M0_target))
    
    # Perform intenrventions in the abstracted model and setup the inference engine
    M1do = A.M1.do(M1_source)
    inferM1 = VariableElimination(M1do)
    
    # Perform intenrventions in the base model and setup the inference engine
    M0do = A.M0.do(M0_source)
    inferM0 = VariableElimination(M0do)
    
    # Evaluate the mechanism in the abstracted model
    M1_joint_TS = inferM1.query(M1_target+M1_source,show_progress=False)
    M1_joint_S = inferM1.query(M1_source,show_progress=False)
    M1_cond_TS = M1_joint_TS / M1_joint_S
    
    # Extract the matrix
    M1_cond_TS_val = M1_cond_TS.values

    # Check ordering
    if (M1_cond_TS.variables[0] != M1_target[0]):
        M1_cond_TS_val = M1_cond_TS_val.T

    # Evaluate the mechanism in the base model
    M0_joint_TS = inferM0.query(M0_target+M0_source,show_progress=False)
    M0_joint_S = inferM0.query(M0_source,show_progress=False)
    M0_cond_TS = M0_joint_TS / M0_joint_S
    
    # Extract the matrix
    M0_cond_TS_val = M0_cond_TS.values

    # Reorder the matrix
    old_indexes = range(len(M0_target+M0_source))
    new_indexes = [(M0_target+M0_source).index(i) for i in M0_joint_TS.variables]
    M0_cond_TS_val = np.moveaxis(M0_cond_TS_val, old_indexes, new_indexes)

    # Compact the matrix
    M0_target_cards=[A.M0.get_cardinality(t) for t in M0_target]
    M0_target_card = np.prod(M0_target_cards)
    M0_source_cards=[A.M0.get_cardinality(s) for s in M0_source]
    M0_source_card = np.prod(M0_source_cards)
    M0_cond_TS_val = M0_cond_TS_val.reshape(M0_target_card,M0_source_card)
    
    # Extract the alphas
    alpha_S = A.alphas[M1_source[0]]
    alpha_T = A.alphas[M1_target[0]]
    
    # Evaluate the paths on the diagram
    lowerpath = np.dot(M1_cond_TS_val,alpha_S)
    upperpath = np.dot(alpha_T,M0_cond_TS_val)
    
    # Compute abstraction error for every possible intervention
    distances = []
    for c in range(lowerpath.shape[1]):
        distances.append( distance.jensenshannon(lowerpath[:,c],upperpath[:,c]) )
    print('All JS distances: {0}'.format(distances))
    
    # Select the greatest distance over all interventions
    print('\nAbstraction error: {0}'.format(np.max(distances)))
    abstraction_errors.append(np.max(distances))

# Select the greatest distance over all pairs considered
print('\n\nOVERALL ABSTRACTION ERROR: {0}'.format(np.max(abstraction_errors)))

M1: ['Smoking'] -> ['Cancer']
M0: ['Smoking'] -> ['Cancer']
All JS distances: [0.0, 4.344397704933063e-09]

Abstraction error: 4.344397704933063e-09


OVERALL ABSTRACTION ERROR: 4.344397704933063e-09

As we knew, this example is a case of zero-error abstraction, and indeed our result is virtually zero once we account for numerical approximation.

Evaluating abstraction considering all pairs of nodes¶

We now move on using the same algorithm with a richer evaluation set $\mathcal{J}$ containg all the pairs $(X,Y)$ in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$, independently from the existence of a directed path.

Example 2¶

We still rely on the prototypical case study on the lung cancer scenario. This time we take into consideration the last extreme abstraction that removes all edges from the abstracted model. If we were to consider an evaluation set $\mathcal{J}$ that contains only directed paths in the abstracted model $\mathcal{M}''$/$\mathtt{M2}$ there would be no path; however in the notebook Categorical Abstraction.ipynb we computed a distance even in the absence of a directed path. For this reason in this example we enlarge the evaluation set $\mathcal{J}$.

Model definition¶

First, we define the models and the abstraction.

In [12]:

M0 = BN([('Smoking','Cancer')])

cpdS = cpd(variable='Smoking',
          variable_card=2,
          values=[[.8],[.2]],
          evidence=None,
          evidence_card=None)
cpdC = cpd(variable='Cancer',
          variable_card=2,
          values=[[.9,.66],[.1,.34]],
          evidence=['Smoking'],
          evidence_card=[2])

M0.add_cpds(cpdS,cpdC)
M0.check_model()

Out[12]:

True

In [13]:

M1 = BN()
M1.add_node('Smoking')
M1.add_node('Cancer')

cpdS = cpd(variable='Smoking',
          variable_card=2,
          values=[[.8],[.2]],
          evidence=None,
          evidence_card=None)
cpdC = cpd(variable='Cancer',
          variable_card=2,
          values=[[.852],[.148]],
          evidence=None,
          evidence_card=None)

M1.add_cpds(cpdS,cpdC)
M1.check_model()

Out[13]:

True

In [14]:

R = ['Smoking','Cancer']

a = {'Smoking': 'Smoking',
    'Cancer': 'Cancer'}
alphas = {'Smoking': np.eye(2),
         'Cancer': np.eye(2)}

In [15]:

A = Abstraction(M0,M1,R,a,alphas)

Enumeration of all possible pairs in M1¶

Then we populate the evaluation set $\mathcal{J}$ consider all possible pairs $(X,Y)$ in the abstracted model $\mathcal{M'}/\mathtt{M1}$. To do so, we just call itertools.permutations() to retrieve all the permutations of two elements.

In [16]:

J = list(itertools.permutations(A.M1.nodes(),2))

In [17]:

print(J)

[('Smoking', 'Cancer'), ('Cancer', 'Smoking')]

Notice that we consider the pair $(Smoking, Cancer)$ even if in the abstracted model there is no link between them. Moreover we consider also the pair $(Cancer, Smoking)$ although it is debatable whether we should consider this inverted relation that requires estimating a mechanism $\mathcal{M}[\phi_{Smoking}] = P(Smoking \vert Cancer)$ that hardly has a causal interpretation.

Running the computational loop¶

We run the same evaluatin loop. See above for an explanation of every step.

In [18]:

abstraction_errors = []

for pair in J:
    # Get nodes in the abstracted model
    M1_source = [pair[0]]
    M1_target = [pair[1]]
    print('\nM1: {0} -> {1}'.format(M1_source,M1_target))

    # Get nodes in the base model
    M0_source = A.invert_a(M1_source)
    M0_target = A.invert_a(M1_target)
    print('M0: {0} -> {1}'.format(M0_source,M0_target))
    
    # Perform intenrventions in the abstracted model and setup the inference engine
    M1do = A.M1.do(M1_source)
    inferM1 = VariableElimination(M1do)
    
    # Perform intenrventions in the base model and setup the inference engine
    M0do = A.M0.do(M0_source)
    inferM0 = VariableElimination(M0do)
    
    # Evaluate the mechanism in the abstracted model
    M1_joint_TS = inferM1.query(M1_target+M1_source,show_progress=False)
    M1_joint_S = inferM1.query(M1_source,show_progress=False)
    M1_cond_TS = M1_joint_TS / M1_joint_S
    
    # Extract the matrix
    M1_cond_TS_val = M1_cond_TS.values

    # Check ordering
    if (M1_cond_TS.variables[0] != M1_target[0]):
        M1_cond_TS_val = M1_cond_TS_val.T

    # Evaluate the mechanism in the base model
    M0_joint_TS = inferM0.query(M0_target+M0_source,show_progress=False)
    M0_joint_S = inferM0.query(M0_source,show_progress=False)
    M0_cond_TS = M0_joint_TS / M0_joint_S
    
    # Extract the matrix
    M0_cond_TS_val = M0_cond_TS.values

    # Reorder the matrix
    old_indexes = range(len(M0_target+M0_source))
    new_indexes = [(M0_target+M0_source).index(i) for i in M0_joint_TS.variables]
    M0_cond_TS_val = np.moveaxis(M0_cond_TS_val, old_indexes, new_indexes)

    # Compact the matrix
    M0_target_cards=[A.M0.get_cardinality(t) for t in M0_target]
    M0_target_card = np.prod(M0_target_cards)
    M0_source_cards=[A.M0.get_cardinality(s) for s in M0_source]
    M0_source_card = np.prod(M0_source_cards)
    M0_cond_TS_val = M0_cond_TS_val.reshape(M0_target_card,M0_source_card)

    # Extract the alphas
    alpha_S = A.alphas[M1_source[0]]
    alpha_T = A.alphas[M1_target[0]]
    
    # Evaluate the paths on the diagram
    lowerpath = np.dot(M1_cond_TS_val,alpha_S)
    upperpath = np.dot(alpha_T,M0_cond_TS_val)
    
    # Compute abstraction error for every possible intervention
    distances = []
    for c in range(lowerpath.shape[1]):
        distances.append( distance.jensenshannon(lowerpath[:,c],upperpath[:,c]) )
    print('All JS distances: {0}'.format(distances))
    
    # Select the greatest distance over all interventions
    print('\nAbstraction error: {0}'.format(np.max(distances)))
    abstraction_errors.append(np.max(distances))

# Select the greatest distance over all pairs considered
print('\n\nOVERALL ABSTRACTION ERROR: {0}'.format(np.max(abstraction_errors)))

M1: ['Smoking'] -> ['Cancer']
M0: ['Smoking'] -> ['Cancer']
All JS distances: [0.051634404110767126, 0.15974085850231143]

Abstraction error: 0.15974085850231143

M1: ['Cancer'] -> ['Smoking']
M0: ['Cancer'] -> ['Smoking']
All JS distances: [0.0, 0.0]

Abstraction error: 0.0


OVERALL ABSTRACTION ERROR: 0.15974085850231143

Notice that, looking at the pair $(Smoking,Cancer)$ we obtain a result in line with what we obtained in the notebook Categorical Abstraction.ipynb. The result for the pair $(Cancer,Smoking)$ may look at first counterintuive: why the error is in this case equal to 0? The result actually make sense because we are measuring $P(Smoking\vert do(Cancer))$ in the two models $\mathcal{M}/M0$ and $\mathcal{M'}/M1$. In $\mathcal{M'}/M1$, $Smoking$ and $Cancer$ are independent from the beginning; in $\mathcal{M}/M0$, $Smoking$ and $Cancer$ are made independent by the intervention. So we are just comparing $P(Smoking)$ in $\mathcal{M}/M0$ and $\mathcal{M'}/M1$, and this marginal happens to have been defined equally in the two models.

Notice, also, that if we were to print $\mathcal{M'}[\phi_{Cancer}] = P(Cancer \vert Smoking)$ (or $\mathcal{M'}[\phi_{Smoking}] = P(Smoking \vert Cancer)$) we could see from its matrix form that it encodes a conditional distribution in which $Smoking$ and $Cancer$ are independent in the abstracted model $\mathcal{M'}/M1$. The matrix has the expected shape $[2,2]$ but it is columns are identical, suggesting that the value on the column does not matter. This makes sense ($Smoking$ and $Cancer$ have no connection in the graph, they are independent), and it is consistent with the formalism and result in the notebook Categorical Abstraction.ipynb.

Evaluating abstraction considering pairs of node potentially having a causal relation¶

Considering an evaluation set $\mathcal{J}$ containing all the pairs $(X,Y)$ in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$ seems to generate a set inclusing meaninglss pairs. We will then refine the evaluation set $\mathcal{J}$ containg all the pairs $(X,Y)$ that are connected by a directed path in $\mathcal{M}'$/$\mathtt{M1}$ or whose counterimages $(\mathbf{A},\mathbf{B})$ are connected by a directed path in $\mathcal{M}$/$\mathtt{M0}$.

Example 3¶

We instantiate now a new test models containing more nodes and more challenging mappings.

Model definition¶

First, we define the models and the abstraction.

In [19]:

def generate_values(c,d):
    val = np.random.rand(c,d)
    return val / np.sum(val,axis=0)

In [20]:

M0 = BN([('A','C'), ('E','B'), ('E','C'), ('B','C'), ('C','D'), ('C','F')])

cpdA = cpd(variable='A',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdE = cpd(variable='E',
          variable_card=2,
          values=generate_values(2,1),
          evidence=None,
          evidence_card=None)

cpdB = cpd(variable='B',
          variable_card=4,
          values=generate_values(4,2),
          evidence=['E'],
          evidence_card=[2])

cpdC = cpd(variable='C',
          variable_card=6,
          values=generate_values(6,24),
          evidence=['A','B','E'],
          evidence_card=[3,4,2])

cpdD = cpd(variable='D',
          variable_card=3,
          values=generate_values(3,6),
          evidence=['C'],
          evidence_card=[6])

cpdF = cpd(variable='F',
          variable_card=2,
          values=generate_values(2,6),
          evidence=['C'],
          evidence_card=[6])

M0.add_cpds(cpdA,cpdB,cpdC,cpdD,cpdE,cpdF)
M0.check_model()

Out[20]:

True

In [21]:

M1 = BN([('X','Y'), ('Y','Z'), ('Y','W'), ('X','Z')])

cpdX = cpd(variable='X',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdY = cpd(variable='Y',
          variable_card=4,
          values=generate_values(4,3),
          evidence=['X'],
          evidence_card=[3])

cpdZ = cpd(variable='Z',
          variable_card=2,
          values=generate_values(2,12),
          evidence=['Y','X'],
          evidence_card=[4,3])

cpdW = cpd(variable='W',
          variable_card=2,
          values=generate_values(2,4),
          evidence=['Y'],
          evidence_card=[4])

M1.add_cpds(cpdX,cpdY,cpdZ,cpdW)
M1.check_model()

Out[21]:

True

In [22]:

R = ['A','B', 'C', 'D', 'F']

a = {'A': 'X',
     'B': 'X',
     'C': 'Y',
     'D': 'Z',
     'F': 'W'}
alphas = {'X': generate_values(3,12),
         'Y': generate_values(4,6),
         'Z': generate_values(2,3),
         'W': generate_values(2,2)}

In [23]:

A = Abstraction(M0,M1,R,a,alphas)

Enumeration of all possible pairs in M1 with a directed path connection in M1 or M0¶

We now populate our evaluation set $\mathcal{J}$ computing all possible pairs (source,target) along directed paths in the abstracted model $\mathcal{M'}/\mathtt{M1}$ or in the base model $\mathcal{M'}/\mathtt{M1}$.

We start implementing a helper function check_path_between_sets() to check the existence of a path between two set of nodes $\mathbf{A}$ and $\mathbf{B}$. This function creates an artificial parent $A'$ to all the nodes in $\mathbf{A}$ and an artifical child $B'$ to all the nodes in $\mathbf{B}$ and then we call the networkx function has_path() between $A'$ and $B'$.

In [24]:

def check_path_between_sets(G,sources,targets):
    augmentedG = G.copy()
            
    augmented_s = 'augmented_s_'+str(np.random.randint(10**6))
    augmented_t = 'augmented_t_'+str(np.random.randint(10**6))
    augmentedG.add_node(augmented_s)
    augmentedG.add_node(augmented_t)

    [augmentedG.add_edge(augmented_s,s) for s in sources]
    [augmentedG.add_edge(t,augmented_t) for t in targets]

    return nx.has_path(augmentedG,augmented_s,augmented_t)  

We can populate $\mathcal{J}$. To verify if there is a path in $\mathcal{M'}/\mathtt{M1}$ between node $X$ and $Y$ we use the same method we used before of calling the networkx function has_path(). To verify if there is a path in $\mathcal{M}/\mathtt{M0}$ between the set of nodes $\mathbf{A}$ and $\mathbf{B}$ we rely on our helper function.

In [25]:

J = []

sources = list(A.M1.nodes())
targets = list(A.M1.nodes())

for s in sources:
    for t in list(set(targets)-{s}):
        if nx.has_path(A.M1,s,t):
            J.append((s,t))
        else:
            M0_sources = A.invert_a(s)
            M0_targets = A.invert_a(t)            
            if check_path_between_sets(A.M0,M0_sources,M0_targets):
                J.append((s,t))

In [26]:

print(J)

[('X', 'W'), ('X', 'Y'), ('X', 'Z'), ('Y', 'W'), ('Y', 'Z')]

Running the computational loop¶

In [27]:

abstraction_errors = []

for pair in J:
    # Get nodes in the abstracted model
    M1_source = [pair[0]]
    M1_target = [pair[1]]
    print('\nM1: {0} -> {1}'.format(M1_source,M1_target))

    # Get nodes in the base model
    M0_source = A.invert_a(M1_source)
    M0_target = A.invert_a(M1_target)
    print('M0: {0} -> {1}'.format(M0_source,M0_target))
    
    # Perform intenrventions in the abstracted model and setup the inference engine
    M1do = A.M1.do(M1_source)
    inferM1 = VariableElimination(M1do)
    
    # Perform intenrventions in the base model and setup the inference engine
    M0do = A.M0.do(M0_source)
    inferM0 = VariableElimination(M0do)
    
    # Evaluate the mechanism in the abstracted model
    M1_joint_TS = inferM1.query(M1_target+M1_source,show_progress=False)
    M1_joint_S = inferM1.query(M1_source,show_progress=False)
    M1_cond_TS = M1_joint_TS / M1_joint_S
    
    # Extract the matrix
    M1_cond_TS_val = M1_cond_TS.values

    # Check ordering
    if (M1_cond_TS.variables[0] != M1_target[0]):
        M1_cond_TS_val = M1_cond_TS_val.T

    # Evaluate the mechanism in the base model
    M0_joint_TS = inferM0.query(M0_target+M0_source,show_progress=False)
    M0_joint_S = inferM0.query(M0_source,show_progress=False)
    M0_cond_TS = M0_joint_TS / M0_joint_S
    
    # Extract the matrix
    M0_cond_TS_val = M0_cond_TS.values

    # Reorder the matrix
    old_indexes = range(len(M0_target+M0_source))
    new_indexes = [(M0_target+M0_source).index(i) for i in M0_joint_TS.variables]
    M0_cond_TS_val = np.moveaxis(M0_cond_TS_val, old_indexes, new_indexes)

    # Compact the matrix
    M0_target_cards=[A.M0.get_cardinality(t) for t in M0_target]
    M0_target_card = np.prod(M0_target_cards)
    M0_source_cards=[A.M0.get_cardinality(s) for s in M0_source]
    M0_source_card = np.prod(M0_source_cards)
    M0_cond_TS_val = M0_cond_TS_val.reshape(M0_target_card,M0_source_card)

    # Extract the alphas
    alpha_S = A.alphas[M1_source[0]]
    alpha_T = A.alphas[M1_target[0]]
    
    # Evaluate the paths on the diagram
    lowerpath = np.dot(M1_cond_TS_val,alpha_S)
    upperpath = np.dot(alpha_T,M0_cond_TS_val)
    
    # Compute abstraction error for every possible intervention
    distances = []
    for c in range(lowerpath.shape[1]):
        distances.append( distance.jensenshannon(lowerpath[:,c],upperpath[:,c]) )
    print('All JS distances: {0}'.format(distances))
    
    # Select the greatest distance over all interventions
    print('\nAbstraction error: {0}'.format(np.max(distances)))
    abstraction_errors.append(np.max(distances))

# Select the greatest distance over all pairs considered
print('\n\nOVERALL ABSTRACTION ERROR: {0}'.format(np.max(abstraction_errors)))

M1: ['X'] -> ['W']
M0: ['A', 'B'] -> ['F']
All JS distances: [0.12521771836868564, 0.1098170956238661, 0.1093643467805497, 0.15438710717432574, 0.12046159176748646, 0.16917931812906775, 0.13779348059871385, 0.1727573437923306, 0.17731904977048296, 0.1549678911567308, 0.13311969480754737, 0.14345170601798582]

Abstraction error: 0.17731904977048296

M1: ['X'] -> ['Y']
M0: ['A', 'B'] -> ['C']
All JS distances: [0.11835877298107207, 0.19699602007864722, 0.09476904663959261, 0.20950728465444696, 0.15927935305699015, 0.14207969276805765, 0.11968508852041435, 0.1564581742712215, 0.17109467359446529, 0.14402290823110306, 0.1451752260206426, 0.15053887286820122]

Abstraction error: 0.20950728465444696

M1: ['X'] -> ['Z']
M0: ['A', 'B'] -> ['D']
All JS distances: [0.00906527140205899, 0.018891757270061746, 0.004833781584752415, 0.02302667236729111, 0.010767043207948444, 0.029256445382254467, 0.014593949677328466, 0.04728920146648553, 0.03818884145805786, 0.021735512048527937, 0.016418485883169343, 0.015928361507320885]

Abstraction error: 0.04728920146648553

M1: ['Y'] -> ['W']
M0: ['C'] -> ['F']
All JS distances: [0.225156509353865, 0.117924866640986, 0.14683158337181426, 0.25848110864747187, 0.12505841562696274, 0.03694187983853757]

Abstraction error: 0.25848110864747187

M1: ['Y'] -> ['Z']
M0: ['C'] -> ['D']
All JS distances: [0.0452083112500307, 0.07283988629719863, 0.013379590628713857, 0.11320708794598638, 0.05004668669378092, 0.028591738846108373]

Abstraction error: 0.11320708794598638


OVERALL ABSTRACTION ERROR: 0.25848110864747187

Evaluating abstraction considering sets of node potentially having a causal relation¶

The actual definition of abstraction error requires considering not only all potential pairs $(X,Y)$ in the abstracted model $\mathcal{M}'$/$\mathtt{M1}$, but every disjoint set of variables $(\mathbf{X},\mathbf{Y})$ in $\mathcal{M}'$/$\mathtt{M1}$. Here, we will populate the evaluation set $\mathcal{J}$ containg with all the sets $(\mathbf{X},\mathbf{Y})$ that are connected by a directed path in $\mathcal{M}'$/$\mathtt{M1}$ or whose counterimages $(\mathbf{A},\mathbf{B})$ are connected by a directed path in $\mathcal{M}$/$\mathtt{M0}$.

Example 4¶

We instantiate the same models as in Example 3.

Model definition¶

We redefine the models and the abstraction.

In [28]:

M0 = BN([('A','C'), ('E','B'), ('E','C'), ('B','C'), ('C','D'), ('C','F')])

cpdA = cpd(variable='A',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdE = cpd(variable='E',
          variable_card=2,
          values=generate_values(2,1),
          evidence=None,
          evidence_card=None)

cpdB = cpd(variable='B',
          variable_card=4,
          values=generate_values(4,2),
          evidence=['E'],
          evidence_card=[2])

cpdC = cpd(variable='C',
          variable_card=6,
          values=generate_values(6,24),
          evidence=['A','B','E'],
          evidence_card=[3,4,2])

cpdD = cpd(variable='D',
          variable_card=3,
          values=generate_values(3,6),
          evidence=['C'],
          evidence_card=[6])

cpdF = cpd(variable='F',
          variable_card=2,
          values=generate_values(2,6),
          evidence=['C'],
          evidence_card=[6])

M0.add_cpds(cpdA,cpdB,cpdC,cpdD,cpdE,cpdF)
M0.check_model()

Out[28]:

True

In [29]:

M1 = BN([('X','Y'), ('Y','Z'), ('Y','W'), ('X','Z')])

cpdX = cpd(variable='X',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdY = cpd(variable='Y',
          variable_card=4,
          values=generate_values(4,3),
          evidence=['X'],
          evidence_card=[3])

cpdZ = cpd(variable='Z',
          variable_card=2,
          values=generate_values(2,12),
          evidence=['Y','X'],
          evidence_card=[4,3])

cpdW = cpd(variable='W',
          variable_card=2,
          values=generate_values(2,4),
          evidence=['Y'],
          evidence_card=[4])

M1.add_cpds(cpdX,cpdY,cpdZ,cpdW)
M1.check_model()

Out[29]:

True

In [30]:

R = ['A','B', 'C', 'D', 'F']

a = {'A': 'X',
     'B': 'X',
     'C': 'Y',
     'D': 'Z',
     'F': 'W'}
alphas = {'X': generate_values(3,12),
         'Y': generate_values(4,6),
         'Z': generate_values(2,3),
         'W': generate_values(2,2)}

In [31]:

A = Abstraction(M0,M1,R,a,alphas)

Enumeration of all possible sets in M1 with directed path connections in M1 or M0¶

We now want to populate our evaluation set $\mathcal{J}$ considering not only pairs of nodes in M1, but all possible ways of generating two disjoint subsets out of $\mathcal{X}_\mathcal{M'}$.

To do so, we start implementing a simple function powerset() to compute the powersets of $\mathcal{X}_\mathcal{M'}$ following the recipe provided in the itertools documentation.

In [32]:

def powerset(iterable):
    s = list(iterable)
    return itertools.chain.from_iterable(itertools.combinations(s, r) for r in range(len(s)+1))

We then populate the evaluation set $\mathcal{J}$ according to the following step:

Consider a pair of sets from the powerset
Check whether the sets are disjoint:
- If yes, move to step 3
- If not, go back to step 1
Check if the two sets of nodes have a directed path in \mathtt{M1}:
- If yes, add them to $\mathcal{J}$ and go back to step 1
- If not, go to step 4
Find the corresponding sets in M0 (Notice that since $a$ is a function we already know these two sets will also be disjoint)
Check if the two sets of nodes have a directed path in \mathtt{M0}:
- If yes, add them to $\mathcal{J}$ and go back to step 1
- If not, go back to step 1

This procedure evaluate all possible pairings of elements in the powerset, and it selects only the right ones for the evaluation set $\mathcal{J}$.

In [33]:

J = []

sets = list(powerset(A.M1.nodes()))
sets.remove(())

for i in sets:
    for j in sets:
        M1_sources = list(i)
        M1_targets = list(j)
        if not(any(x in M1_sources for x in M1_targets)):            
            if check_path_between_sets(A.M1,M1_sources,M1_targets):
                print('- Checking {0} -> {1}: True'.format(M1_sources,M1_targets))
                J.append([M1_sources,M1_targets])
            else:
                print('- Checking {0} -> {1}: False'.format(M1_sources,M1_targets))
                M0_sources = A.invert_a(M1_sources)
                M0_targets = A.invert_a(M1_targets)
                if check_path_between_sets(A.M0,M0_sources,M0_targets):
                    print('---- Checking {0} -> {1}: True'.format(M0_sources,M0_targets))
                    J.append([M1_sources,M1_targets])
                else:
                    print('---- Checking {0} -> {1}: False'.format(M0_sources,M0_targets))
                        
print('\n {0} legitimate pairs of sets out of {1} possbile pairs of sets'.format(len(J),len(sets)**2))    

- Checking ['X'] -> ['Y']: True
- Checking ['X'] -> ['Z']: True
- Checking ['X'] -> ['W']: True
- Checking ['X'] -> ['Y', 'Z']: True
- Checking ['X'] -> ['Y', 'W']: True
- Checking ['X'] -> ['Z', 'W']: True
- Checking ['X'] -> ['Y', 'Z', 'W']: True
- Checking ['Y'] -> ['X']: False
---- Checking ['C'] -> ['A', 'B']: False
- Checking ['Y'] -> ['Z']: True
- Checking ['Y'] -> ['W']: True
- Checking ['Y'] -> ['X', 'Z']: True
- Checking ['Y'] -> ['X', 'W']: True
- Checking ['Y'] -> ['Z', 'W']: True
- Checking ['Y'] -> ['X', 'Z', 'W']: True
- Checking ['Z'] -> ['X']: False
---- Checking ['D'] -> ['A', 'B']: False
- Checking ['Z'] -> ['Y']: False
---- Checking ['D'] -> ['C']: False
- Checking ['Z'] -> ['W']: False
---- Checking ['D'] -> ['F']: False
- Checking ['Z'] -> ['X', 'Y']: False
---- Checking ['D'] -> ['A', 'B', 'C']: False
- Checking ['Z'] -> ['X', 'W']: False
---- Checking ['D'] -> ['A', 'B', 'F']: False
- Checking ['Z'] -> ['Y', 'W']: False
---- Checking ['D'] -> ['C', 'F']: False
- Checking ['Z'] -> ['X', 'Y', 'W']: False
---- Checking ['D'] -> ['A', 'B', 'C', 'F']: False
- Checking ['W'] -> ['X']: False
---- Checking ['F'] -> ['A', 'B']: False
- Checking ['W'] -> ['Y']: False
---- Checking ['F'] -> ['C']: False
- Checking ['W'] -> ['Z']: False
---- Checking ['F'] -> ['D']: False
- Checking ['W'] -> ['X', 'Y']: False
---- Checking ['F'] -> ['A', 'B', 'C']: False
- Checking ['W'] -> ['X', 'Z']: False
---- Checking ['F'] -> ['A', 'B', 'D']: False
- Checking ['W'] -> ['Y', 'Z']: False
---- Checking ['F'] -> ['C', 'D']: False
- Checking ['W'] -> ['X', 'Y', 'Z']: False
---- Checking ['F'] -> ['A', 'B', 'C', 'D']: False
- Checking ['X', 'Y'] -> ['Z']: True
- Checking ['X', 'Y'] -> ['W']: True
- Checking ['X', 'Y'] -> ['Z', 'W']: True
- Checking ['X', 'Z'] -> ['Y']: True
- Checking ['X', 'Z'] -> ['W']: True
- Checking ['X', 'Z'] -> ['Y', 'W']: True
- Checking ['X', 'W'] -> ['Y']: True
- Checking ['X', 'W'] -> ['Z']: True
- Checking ['X', 'W'] -> ['Y', 'Z']: True
- Checking ['Y', 'Z'] -> ['X']: False
---- Checking ['C', 'D'] -> ['A', 'B']: False
- Checking ['Y', 'Z'] -> ['W']: True
- Checking ['Y', 'Z'] -> ['X', 'W']: True
- Checking ['Y', 'W'] -> ['X']: False
---- Checking ['C', 'F'] -> ['A', 'B']: False
- Checking ['Y', 'W'] -> ['Z']: True
- Checking ['Y', 'W'] -> ['X', 'Z']: True
- Checking ['Z', 'W'] -> ['X']: False
---- Checking ['D', 'F'] -> ['A', 'B']: False
- Checking ['Z', 'W'] -> ['Y']: False
---- Checking ['D', 'F'] -> ['C']: False
- Checking ['Z', 'W'] -> ['X', 'Y']: False
---- Checking ['D', 'F'] -> ['A', 'B', 'C']: False
- Checking ['X', 'Y', 'Z'] -> ['W']: True
- Checking ['X', 'Y', 'W'] -> ['Z']: True
- Checking ['X', 'Z', 'W'] -> ['Y']: True
- Checking ['Y', 'Z', 'W'] -> ['X']: False
---- Checking ['C', 'D', 'F'] -> ['A', 'B']: False

 29 legitimate pairs of sets out of 225 possbile pairs of sets

In [34]:

print(J)

[[['X'], ['Y']], [['X'], ['Z']], [['X'], ['W']], [['X'], ['Y', 'Z']], [['X'], ['Y', 'W']], [['X'], ['Z', 'W']], [['X'], ['Y', 'Z', 'W']], [['Y'], ['Z']], [['Y'], ['W']], [['Y'], ['X', 'Z']], [['Y'], ['X', 'W']], [['Y'], ['Z', 'W']], [['Y'], ['X', 'Z', 'W']], [['X', 'Y'], ['Z']], [['X', 'Y'], ['W']], [['X', 'Y'], ['Z', 'W']], [['X', 'Z'], ['Y']], [['X', 'Z'], ['W']], [['X', 'Z'], ['Y', 'W']], [['X', 'W'], ['Y']], [['X', 'W'], ['Z']], [['X', 'W'], ['Y', 'Z']], [['Y', 'Z'], ['W']], [['Y', 'Z'], ['X', 'W']], [['Y', 'W'], ['Z']], [['Y', 'W'], ['X', 'Z']], [['X', 'Y', 'Z'], ['W']], [['X', 'Y', 'W'], ['Z']], [['X', 'Z', 'W'], ['Y']]]

Running the computational loop¶

Running the computational loop is conceptually equivalent to what we have done so far, but we have now to deal with the fact that also when working within the high-level model we may have to consider sets of variables and not just individual nodes.

Instead of the diagram we have seen before:

we have to consider:

$$ \begin{array}{ccc} \mathcal{\mathcal{M}_{do}}\left[\mathbf{A}\right] & \overset{\mathcal{\mathcal{M}_{do}}\left[\phi_{\mathbf{B}}\right]}{\longrightarrow} & \mathcal{\mathcal{M}_{do}}\left[\mathbf{B}\right]\\ \sideset{}{\alpha_{\mathbf{X}}}\downarrow & & \sideset{}{\alpha_{\mathbf{Y}}}\downarrow\\ \mathcal{\mathcal{M'}_{do}}\left[\mathbf{X}\right] & \overset{\mathcal{\mathcal{M'}_{do}}\left[\phi_{\mathbf{Y}}\right]}{\longrightarrow} & \mathcal{\mathcal{M'}_{do}}\left[\mathbf{Y}\right] \end{array} $$

where $X$ and $Y$ has now been typed in boldface, $\mathbf{X},\mathbf{Y}$ to denote we are working with sets.

This requires us assemble abstractions ($\alpha_{\mathbf{X}},\alpha_{\mathbf{Y}}$) and mechanisms ($\mathcal{M}[\phi_{\mathbf{Y}}]$) with the proper dimensionality. To do so we introduce two helper functions:

tensorize_list: given a list of matrices $[X_1, X_2, X_3, ..., X_n]$ with dimensions $[(r_1,c_1),(r_2,c_2),(r_3,c_3),...,(r_n,c_n)]$, this function recursively compute the tensor product $X_1 \otimes X_2 \otimes X_3 \otimes ... \otimes X_n$ with dimension $(r_1\cdot r_2\cdot r_3\cdot ...\cdot r_n, c_1\cdot c_2\cdot c_3\cdot ...\cdot c_n)$.
tensorize_mechanism: this function wraps up the code we used previously to compute the mechanism in $\mathtt{M0}$. It receives the inference engine for the underlying graph, a source set of nodes, a target set of nodes, and a dictionary with the cardinalities of the nodes. It then use the engine to evaluate the mechanism, and the it re-orders and compacts the resulting matrix in the right shape.

In [35]:

def tensorize_list(tensor,l):
    if tensor is None:
        if len(l)>1:
            tensor = np.einsum('ij,kl->ikjl',l[0],l[1])
            tensor = tensor.reshape((tensor.shape[0]*tensor.shape[1],tensor.shape[2]*tensor.shape[3]))
            return tensorize_list(tensor,l[2:])
        else:
            return l[0]
    else:
        if len(l)>0:
            tensor = np.einsum('ij,kl->ikjl',tensor,l[0])
            tensor = tensor.reshape((tensor.shape[0]*tensor.shape[1],tensor.shape[2]*tensor.shape[3]))
            return tensorize_list(tensor,l[1:])
        else:
            return tensor

In [36]:

def tensorize_mechanisms(inference,sources,targets,cardinalities):
    
    # Evaluate the mechanism
    joint_TS = inference.query(targets+sources,show_progress=False)
    marginal_S = inference.query(sources,show_progress=False)
    cond_TS = joint_TS / marginal_S
    
    # Extract the matrix
    cond_TS_val = cond_TS.values

    # Reorder the matrix
    old_indexes = range(len(targets+sources))
    new_indexes = [(targets+sources).index(i) for i in joint_TS.variables]
    cond_TS_val = np.moveaxis(cond_TS_val, old_indexes, new_indexes)

    # Compact the matrix
    target_cards=[cardinalities[t] for t in targets]
    target_card = np.prod(target_cards)
    source_cards=[cardinalities[s] for s in sources]
    source_card = np.prod(source_cards)
    cond_TS_val = cond_TS_val.reshape(target_card,source_card)
    
    return cond_TS_val

We can then run the actual computational loop. Its overall organization is the same, although we rely on the functions defined above to make it more compact:

Given a pair from the evaluation set $\mathcal{J}$, extract the source nodes $\mathbf{X}$ and the target nodes $\mathbf{Y}$ in the abstracted model $\mathcal{M'}/\mathtt{M1}$.
Using the mapping $a$, retrieve the corresponding source set $\mathbf{A}$ and target set $\mathbf{B}$ in the base model $\mathcal{M}/\mathtt{M0}$.
Compute the intervention $do(X)$ in the abstracted model $\mathcal{M'}/\mathtt{M1}$ and initialize the pgmpy inference engine on this intervened model.
Compute the intervention $do(\mathbf{A})$ in the base model $\mathcal{M}/\mathtt{M0}$ and initialize the pgmpy inference engine on this intervened model.
Compute the mechanism $\mathcal{M'}_{do}[\phi_\mathbf{Y}]$ relying on tensorize_mechanism().
Compute the mechanism $\mathcal{M}_{do}[\phi_\mathbf{B}]$ relying on tensorize_mechanism().
Compute the matrix for $\alpha_\mathbf{X}$ relying on tensorize_list().
Compute the matrix for $\alpha_\mathbf{Y}$ relying on tensorize_list().
Compute the two alternative path on the diagram by composing $\mathcal{M'}_{do}[\phi_\mathbf{Y}] \circ \alpha_\mathbf{X}$ (lower path) and $\alpha_\mathbf{Y} \circ \mathcal{M}_{do}[\phi_\mathbf{B}]$ (upper path) via a simple matrix product.
For every possible intervention on $\mathbf{A}$ compute the JS distance between the two paths.
Select the highest distance with respect to all the interventions as the error $E_\alpha(\mathbf{Y},\mathbf{X})$.
Select the highest distance with repsect to all pairs $(\mathbf{X},\mathbf{Y})$ as the error $e(\alpha)$.

In [37]:

abstraction_errors = []

for pair in J:
    # Get nodes in the abstracted model
    M1_sources = pair[0]
    M1_targets = pair[1]
    print('\nM1: {0} -> {1}'.format(M1_sources,M1_targets))

    # Get nodes in the base model
    M0_sources = A.invert_a(M1_sources)
    M0_targets = A.invert_a(M1_targets)
    print('M0: {0} -> {1}'.format(M0_sources,M0_targets))
    
    # Perform interventions in the abstracted model and setup the inference engine
    M1do = A.M1.do(M1_sources)
    inferM1 = VariableElimination(M1do)
    
    # Perform interventions in the base model and setup the inference engine
    M0do = A.M0.do(M0_sources)
    inferM0 = VariableElimination(M0do)

    # Compute the high-level mechanisms
    M1_cond_TS_val = tensorize_mechanisms(inferM1,M1_sources,M1_targets,A.M1.get_cardinality())
    print('M1 mechanism shape: {}'.format(M1_cond_TS_val.shape))
        
    # Compute the low-level mechanisms
    M0_cond_TS_val = tensorize_mechanisms(inferM0,M0_sources,M0_targets,A.M0.get_cardinality())
    print('M0 mechanism shape: {}'.format(M0_cond_TS_val.shape))

    # Compute the alpha for sources
    alphas_S = [A.alphas[i] for i in M1_sources]
    alpha_S = tensorize_list(None,alphas_S)
    print('Alpha_s shape: {}'.format(alpha_S.shape))
    
    # Compute the alpha for targers
    alphas_T = [A.alphas[i] for i in M1_targets]
    alpha_T = tensorize_list(None,alphas_T)
    print('Alpha_t shape: {}'.format(alpha_T.shape))
        
    # Evaluate the paths on the diagram
    lowerpath = np.dot(M1_cond_TS_val,alpha_S)
    upperpath = np.dot(alpha_T,M0_cond_TS_val)
    
    # Compute abstraction error for every possible intervention
    distances = []
    for c in range(lowerpath.shape[1]):
        distances.append( distance.jensenshannon(lowerpath[:,c],upperpath[:,c]) )
    print('All JS distances: {0}'.format(distances))
    
    # Select the greatest distance over all interventions
    print('\nAbstraction error: {0}'.format(np.max(distances)))
    abstraction_errors.append(np.max(distances))

# Select the greatest distance over all pairs considered
print('\n\nOVERALL ABSTRACTION ERROR: {0}'.format(np.max(abstraction_errors)))

M1: ['X'] -> ['Y']
M0: ['A', 'B'] -> ['C']
M1 mechanism shape: (4, 3)
M0 mechanism shape: (6, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307985, 0.09265621144301286, 0.12001084898732219, 0.15997989391107853, 0.16203192810476508, 0.12696680980769243, 0.10069809180846691, 0.1863697329033639, 0.1047695132817974, 0.11807985631483506, 0.1179253134973884, 0.18437104992516792]

Abstraction error: 0.1863697329033639

M1: ['X'] -> ['Z']
M0: ['A', 'B'] -> ['D']
M1 mechanism shape: (2, 3)
M0 mechanism shape: (3, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (2, 3)
All JS distances: [0.0351304898487227, 0.06733630947989032, 0.0313707716587073, 0.0329667651770552, 0.03126112349484746, 0.03158249424275995, 0.047243008500498236, 0.0019017729573616153, 0.03424179787827428, 0.02209397224515851, 0.023474779635804806, 0.04781008084384352]

Abstraction error: 0.06733630947989032

M1: ['X'] -> ['W']
M0: ['A', 'B'] -> ['F']
M1 mechanism shape: (2, 3)
M0 mechanism shape: (2, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (2, 2)
All JS distances: [0.030672034914896472, 0.045551698121225746, 0.047482514947993826, 0.040984523781884684, 0.041699048049716914, 0.04609574520099206, 0.04539146222920609, 0.04707914958025027, 0.04617497146167864, 0.041023811525147724, 0.044405940952980685, 0.04921460205222507]

Abstraction error: 0.04921460205222507

M1: ['X'] -> ['Y', 'Z']
M0: ['A', 'B'] -> ['C', 'D']
M1 mechanism shape: (8, 3)
M0 mechanism shape: (18, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (8, 18)
All JS distances: [0.18714930747588385, 0.16342782470143777, 0.13923688032004672, 0.16996752417713565, 0.17244633676294255, 0.15413332692010762, 0.15329859180094013, 0.20291390546039728, 0.11960094914967821, 0.12933417631167846, 0.12854231172535505, 0.22176385875401214]

Abstraction error: 0.22176385875401214

M1: ['X'] -> ['Y', 'W']
M0: ['A', 'B'] -> ['C', 'F']
M1 mechanism shape: (8, 3)
M0 mechanism shape: (12, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (8, 12)
All JS distances: [0.15674163338369104, 0.11027827540763376, 0.1344765032870174, 0.16900387860096566, 0.17132352601049652, 0.13971800215379107, 0.11679126446490122, 0.19539161442730005, 0.12050680001965054, 0.13092227531421416, 0.13112660874946497, 0.19297402947863096]

Abstraction error: 0.19539161442730005

M1: ['X'] -> ['Z', 'W']
M0: ['A', 'B'] -> ['D', 'F']
M1 mechanism shape: (4, 3)
M0 mechanism shape: (6, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (4, 6)
All JS distances: [0.047192705325690115, 0.08127081190113358, 0.05706325256989926, 0.0529727107172862, 0.052399220810823675, 0.055975864083481014, 0.06555189636445083, 0.047829683686611724, 0.05780309803964864, 0.04684660609816745, 0.05055104512955366, 0.06868965197737846]

Abstraction error: 0.08127081190113358

M1: ['X'] -> ['Y', 'Z', 'W']
M0: ['A', 'B'] -> ['C', 'D', 'F']
M1 mechanism shape: (16, 3)
M0 mechanism shape: (36, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (16, 36)
All JS distances: [0.1936393333585678, 0.17380671152417954, 0.1517981935295362, 0.178458587800174, 0.1811617242150942, 0.16470162425368473, 0.16415818702211926, 0.2111959555128114, 0.1335662377117007, 0.14111186401600812, 0.1407190157079215, 0.22883249205446551]

Abstraction error: 0.22883249205446551

M1: ['Y'] -> ['Z']
M0: ['C'] -> ['D']
M1 mechanism shape: (2, 4)
M0 mechanism shape: (3, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (2, 3)
All JS distances: [0.06099916587205272, 0.032682775053057354, 0.04679729540729152, 0.07128377077244254, 0.049598194647336426, 0.07518600631595818]

Abstraction error: 0.07518600631595818

M1: ['Y'] -> ['W']
M0: ['C'] -> ['F']
M1 mechanism shape: (2, 4)
M0 mechanism shape: (2, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387506, 0.047747995033239246, 0.04670317536074079, 0.05886660312881184, 0.04156271762404307, 0.030013746023442218]

Abstraction error: 0.10227293864387506

M1: ['Y'] -> ['X', 'Z']
M0: ['C'] -> ['A', 'B', 'D']
M1 mechanism shape: (6, 4)
M0 mechanism shape: (36, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (6, 36)
All JS distances: [0.17389072631509284, 0.14619836813152523, 0.19798973566307185, 0.19883946724488175, 0.16388446559385356, 0.16929406716985493]

Abstraction error: 0.19883946724488175

M1: ['Y'] -> ['X', 'W']
M0: ['C'] -> ['A', 'B', 'F']
M1 mechanism shape: (6, 4)
M0 mechanism shape: (24, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (6, 24)
All JS distances: [0.1688062382679865, 0.14357083895353628, 0.14323935375592675, 0.14749571686086949, 0.141704732098744, 0.13886536590930515]

Abstraction error: 0.1688062382679865

M1: ['Y'] -> ['Z', 'W']
M0: ['C'] -> ['D', 'F']
M1 mechanism shape: (4, 4)
M0 mechanism shape: (6, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (4, 6)
All JS distances: [0.1187091362816008, 0.05814390050461374, 0.06633832145879023, 0.09232883649684216, 0.06498429821122392, 0.08105295591377457]

Abstraction error: 0.1187091362816008

M1: ['Y'] -> ['X', 'Z', 'W']
M0: ['C'] -> ['A', 'B', 'D', 'F']
M1 mechanism shape: (12, 4)
M0 mechanism shape: (72, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (12, 72)
All JS distances: [0.19992799783427462, 0.15363108648024623, 0.2032521275291327, 0.20662976317432394, 0.16893301270801445, 0.17187442878295403]

Abstraction error: 0.20662976317432394

M1: ['X', 'Y'] -> ['Z']
M0: ['A', 'B', 'C'] -> ['D']
M1 mechanism shape: (2, 12)
M0 mechanism shape: (3, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (2, 3)
All JS distances: [0.025789602635587733, 0.01928760783671703, 0.12310884042729682, 0.07219191193089779, 0.05594124826373502, 0.01466139265572651, 0.09004690651961561, 0.04826356728265454, 0.09547683896743701, 0.11540183227374494, 0.07826351680322807, 0.10131564510398797, 0.053601657924777175, 0.02941927906020033, 0.03941675236485532, 0.06259599591646454, 0.045639070821855615, 0.07043791793351223, 0.05139084057708265, 0.022870742231003844, 0.001237585378362012, 0.041848909725622566, 0.019424630254393963, 0.05460400197138135, 0.05599715696418146, 0.027122467588032502, 0.020005636372832054, 0.05442277998472925, 0.031810856733685486, 0.06328039368059145, 0.055445147088796, 0.03213037716307588, 0.05372437885798094, 0.07100067370421342, 0.055219736073434514, 0.07655233188728902, 0.07028672642965786, 0.03980667043736547, 0.07636409101905753, 0.09236313112750567, 0.06840199705104537, 0.08907739020327865, 0.01519831469291378, 0.00015322525000538317, 0.07607525073510538, 0.021262942057825262, 0.029347575786980565, 0.014404367914743779, 0.04594724694162721, 0.02144680588582337, 0.0022106708412245294, 0.03871882685939943, 0.02093299844177826, 0.053661554249806453, 0.041557830291206355, 0.022076302224551918, 0.014443698689199013, 0.0419614662685693, 0.0301451872456355, 0.05749903766054175, 0.04074699969646032, 0.019581160259392962, 6.842138417617628e-05, 0.03410170063702787, 0.020133384916813008, 0.0514646603133323, 0.033576013822963664, 0.022633868883736682, 0.12984449939636644, 0.08079336657139817, 0.059414719557766074, 0.019282712163695793]

Abstraction error: 0.12984449939636644

M1: ['X', 'Y'] -> ['W']
M0: ['A', 'B', 'C'] -> ['F']
M1 mechanism shape: (2, 12)
M0 mechanism shape: (2, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387503, 0.04774799503323982, 0.04670317536074194, 0.058866603128811666, 0.041562717624043385, 0.03001374602344268, 0.10227293864387496, 0.04774799503323982, 0.046703175360741464, 0.0588666031288114, 0.041562717624043385, 0.030013746023443286, 0.10227293864387489, 0.047747995033239864, 0.046703175360741055, 0.05886660312881184, 0.04156271762404307, 0.030013746023441812, 0.102272938643875, 0.047747995033240405, 0.04670317536074079, 0.05886660312881117, 0.041562717624043385, 0.030013746023442273, 0.10227293864387506, 0.04774799503324011, 0.046703175360741055, 0.05886660312881164, 0.04156271762404307, 0.030013746023441784, 0.10227293864387473, 0.047747995033240405, 0.04670317536074109, 0.05886660312881117, 0.041562717624043385, 0.03001374602344372, 0.10227293864387489, 0.04774799503323953, 0.04670317536074172, 0.05886660312881202, 0.04156271762404372, 0.03001374602344184, 0.102272938643875, 0.047747995033239864, 0.04670317536074079, 0.05886660312881214, 0.041562717624043385, 0.030013746023442218, 0.10227293864387496, 0.04774799503323917, 0.04670317536074135, 0.058866603128812256, 0.041562717624043385, 0.030013746023442794, 0.10227293864387534, 0.04774799503323975, 0.04670317536074049, 0.058866603128811666, 0.041562717624043385, 0.030013746023442273, 0.1022729386438754, 0.04774799503324008, 0.046703175360741644, 0.058866603128811666, 0.041562717624042386, 0.030013746023443286, 0.10227293864387506, 0.047747995033240336, 0.04670317536074194, 0.05886660312881193, 0.041562717624043385, 0.030013746023443113]

Abstraction error: 0.1022729386438754

M1: ['X', 'Y'] -> ['Z', 'W']
M0: ['A', 'B', 'C'] -> ['D', 'F']
M1 mechanism shape: (4, 12)
M0 mechanism shape: (6, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (4, 6)
All JS distances: [0.1056624636155618, 0.052084731108216153, 0.1314612858388807, 0.09276636503927882, 0.06956464778069847, 0.03438890194347461, 0.13531096368386336, 0.06803935493658374, 0.10652381872777678, 0.12915427868645055, 0.08881093554411168, 0.1056417634958173, 0.11519772939245851, 0.056367214614913355, 0.06134614046012533, 0.08585858786558867, 0.062008491375707214, 0.07667885627900818, 0.11445924091758652, 0.05348776375112503, 0.046759883610066603, 0.07228635207813679, 0.0462191596341699, 0.0626470708452943, 0.11643580818005946, 0.0553474006039522, 0.050928365869165666, 0.08015841444284152, 0.05266507220301004, 0.07026312381719627, 0.11597516645244803, 0.05776071058397569, 0.07147565036567279, 0.0921053837481447, 0.0693613171125721, 0.08228007437489032, 0.12347415890694341, 0.06232328459792683, 0.0898023235241765, 0.10927070948737225, 0.08025302982041745, 0.09399854331758245, 0.10395604804843733, 0.04856581683701331, 0.08914721232055513, 0.06260268068822607, 0.05093508167652123, 0.03433159933718795, 0.112132560182083, 0.052834658560287, 0.04681448119141199, 0.07052868157394003, 0.04687061220448419, 0.06179520612717915, 0.11034076169497048, 0.052972029514956304, 0.049037825077057624, 0.07233741738004239, 0.05165774297906071, 0.06507406711816113, 0.11013504899786426, 0.05207268084117108, 0.04676312167240142, 0.06811731337283324, 0.04651110833275453, 0.05988487637176444, 0.10768177942575126, 0.05336316759911026, 0.13775997628096487, 0.0995174522023532, 0.07236330825668591, 0.0365645649668503]

Abstraction error: 0.13775997628096487

M1: ['X', 'Z'] -> ['Y']
M0: ['A', 'B', 'D'] -> ['C']
M1 mechanism shape: (4, 6)
M0 mechanism shape: (6, 36)
Alpha_s shape: (6, 36)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307957, 0.1485260632930798, 0.14852606329307955, 0.09265621144301266, 0.09265621144301273, 0.0926562114430127, 0.12001084898732221, 0.12001084898732214, 0.12001084898732231, 0.1599798939110785, 0.15997989391107822, 0.15997989391107834, 0.1620319281047652, 0.16203192810476508, 0.16203192810476508, 0.12696680980769237, 0.1269668098076927, 0.12696680980769257, 0.10069809180846705, 0.10069809180846673, 0.10069809180846714, 0.18636973290336362, 0.18636973290336362, 0.1863697329033639, 0.1047695132817973, 0.10476951328179758, 0.10476951328179743, 0.11807985631483495, 0.11807985631483514, 0.11807985631483495, 0.11792531349738826, 0.11792531349738844, 0.11792531349738834, 0.18437104992516795, 0.18437104992516803, 0.1843710499251681]

Abstraction error: 0.1863697329033639

M1: ['X', 'Z'] -> ['W']
M0: ['A', 'B', 'D'] -> ['F']
M1 mechanism shape: (2, 6)
M0 mechanism shape: (2, 36)
Alpha_s shape: (6, 36)
Alpha_t shape: (2, 2)
All JS distances: [0.03067203491489729, 0.030672034914896472, 0.030672034914896895, 0.045551698121225746, 0.04555169812122603, 0.045551698121225746, 0.04748251494799353, 0.047482514947992945, 0.047482514947992945, 0.0409845237818843, 0.040984523781884684, 0.04098452378188504, 0.041699048049717226, 0.041699048049717226, 0.041699048049717205, 0.04609574520099206, 0.04609574520099146, 0.04609574520099236, 0.04539146222920569, 0.04539146222920508, 0.045391462229205155, 0.04707914958024961, 0.04707914958024957, 0.04707914958024961, 0.04617497146167895, 0.04617497146167951, 0.04617497146167917, 0.04102381152514819, 0.04102381152514891, 0.041023811525147516, 0.04440594095298096, 0.044405940952980685, 0.044405940952980984, 0.04921460205222507, 0.04921460205222507, 0.049214602052225276]

Abstraction error: 0.049214602052225276

M1: ['X', 'Z'] -> ['Y', 'W']
M0: ['A', 'B', 'D'] -> ['C', 'F']
M1 mechanism shape: (8, 6)
M0 mechanism shape: (12, 36)
Alpha_s shape: (6, 36)
Alpha_t shape: (8, 12)
All JS distances: [0.15674163338369085, 0.15674163338369088, 0.15674163338369085, 0.11027827540763371, 0.11027827540763364, 0.11027827540763355, 0.13447650328701752, 0.13447650328701752, 0.1344765032870175, 0.16900387860096577, 0.16900387860096575, 0.16900387860096572, 0.1713235260104964, 0.17132352601049647, 0.17132352601049639, 0.13971800215379096, 0.13971800215379113, 0.13971800215379096, 0.1167912644649011, 0.11679126446490126, 0.11679126446490122, 0.19539161442730002, 0.19539161442730005, 0.19539161442729996, 0.12050680001965042, 0.1205068000196504, 0.12050680001965045, 0.13092227531421435, 0.13092227531421438, 0.13092227531421424, 0.13112660874946505, 0.131126608749465, 0.13112660874946483, 0.19297402947863102, 0.19297402947863096, 0.19297402947863096]

Abstraction error: 0.19539161442730005

M1: ['X', 'W'] -> ['Y']
M0: ['A', 'B', 'F'] -> ['C']
M1 mechanism shape: (4, 6)
M0 mechanism shape: (6, 24)
Alpha_s shape: (6, 24)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307963, 0.14852606329307957, 0.09265621144301267, 0.09265621144301286, 0.12001084898732224, 0.12001084898732221, 0.15997989391107834, 0.15997989391107845, 0.16203192810476497, 0.16203192810476522, 0.12696680980769276, 0.12696680980769245, 0.10069809180846709, 0.10069809180846699, 0.18636973290336376, 0.1863697329033638, 0.10476951328179737, 0.10476951328179754, 0.11807985631483475, 0.11807985631483489, 0.11792531349738826, 0.11792531349738844, 0.18437104992516803, 0.18437104992516806]

Abstraction error: 0.1863697329033638

M1: ['X', 'W'] -> ['Z']
M0: ['A', 'B', 'F'] -> ['D']
M1 mechanism shape: (2, 6)
M0 mechanism shape: (3, 24)
Alpha_s shape: (6, 24)
Alpha_t shape: (2, 3)
All JS distances: [0.035130489848721955, 0.03513048984872272, 0.06733630947989032, 0.06733630947989032, 0.03137077165870871, 0.03137077165870871, 0.03296676517705488, 0.03296676517705488, 0.03126112349484666, 0.03126112349484746, 0.03158249424276041, 0.03158249424276083, 0.04724300850049828, 0.047243008500497945, 0.0019017729573485886, 0.0019017729573485886, 0.034241797878275367, 0.03424179787827465, 0.02209397224515739, 0.02209397224515739, 0.023474779635806173, 0.023474779635806173, 0.04781008084384352, 0.04781008084384316]

Abstraction error: 0.06733630947989032

M1: ['X', 'W'] -> ['Y', 'Z']
M0: ['A', 'B', 'F'] -> ['C', 'D']
M1 mechanism shape: (8, 6)
M0 mechanism shape: (18, 24)
Alpha_s shape: (6, 24)
Alpha_t shape: (8, 18)
All JS distances: [0.18714930747588385, 0.18714930747588387, 0.1634278247014377, 0.16342782470143763, 0.13923688032004664, 0.1392368803200467, 0.16996752417713568, 0.1699675241771357, 0.17244633676294252, 0.17244633676294258, 0.15413332692010767, 0.15413332692010773, 0.15329859180094013, 0.15329859180094002, 0.20291390546039717, 0.20291390546039711, 0.11960094914967835, 0.11960094914967805, 0.12933417631167843, 0.12933417631167834, 0.1285423117253549, 0.1285423117253551, 0.22176385875401208, 0.22176385875401208]

Abstraction error: 0.22176385875401208

M1: ['Y', 'Z'] -> ['W']
M0: ['C', 'D'] -> ['F']
M1 mechanism shape: (2, 8)
M0 mechanism shape: (2, 18)
Alpha_s shape: (8, 18)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387506, 0.10227293864387496, 0.10227293864387489, 0.047747995033239246, 0.04774799503324015, 0.04774799503323957, 0.04670317536074049, 0.04670317536074139, 0.046703175360741464, 0.05886660312881117, 0.05886660312881202, 0.05886660312881202, 0.04156271762404405, 0.04156271762404405, 0.04156271762404372, 0.030013746023442218, 0.030013746023441784, 0.030013746023442218]

Abstraction error: 0.10227293864387506

M1: ['Y', 'Z'] -> ['X', 'W']
M0: ['C', 'D'] -> ['A', 'B', 'F']
M1 mechanism shape: (6, 8)
M0 mechanism shape: (24, 18)
Alpha_s shape: (8, 18)
Alpha_t shape: (6, 24)
All JS distances: [0.1688062382679865, 0.1688062382679864, 0.16880623826798657, 0.14357083895353626, 0.14357083895353628, 0.14357083895353615, 0.1432393537559268, 0.14323935375592697, 0.14323935375592672, 0.14749571686086962, 0.14749571686086962, 0.1474957168608697, 0.14170473209874385, 0.14170473209874374, 0.14170473209874376, 0.13886536590930534, 0.1388653659093051, 0.138865365909305]

Abstraction error: 0.16880623826798657

M1: ['Y', 'W'] -> ['Z']
M0: ['C', 'F'] -> ['D']
M1 mechanism shape: (2, 8)
M0 mechanism shape: (3, 12)
Alpha_s shape: (8, 12)
Alpha_t shape: (2, 3)
All JS distances: [0.060999165872052154, 0.060999165872052154, 0.032682775053056, 0.03268277505305698, 0.046797295407291155, 0.0467972954072918, 0.07128377077244273, 0.07128377077244222, 0.04959819464733618, 0.049598194647336426, 0.07518600631595801, 0.07518600631595769]

Abstraction error: 0.07518600631595801

M1: ['Y', 'W'] -> ['X', 'Z']
M0: ['C', 'F'] -> ['A', 'B', 'D']
M1 mechanism shape: (6, 8)
M0 mechanism shape: (36, 12)
Alpha_s shape: (8, 12)
Alpha_t shape: (6, 36)
All JS distances: [0.17389072631509295, 0.17389072631509261, 0.14619836813152512, 0.1461983681315251, 0.1979897356630717, 0.19798973566307185, 0.19883946724488183, 0.19883946724488177, 0.16388446559385353, 0.16388446559385347, 0.16929406716985484, 0.16929406716985496]

Abstraction error: 0.19883946724488183

M1: ['X', 'Y', 'Z'] -> ['W']
M0: ['A', 'B', 'C', 'D'] -> ['F']
M1 mechanism shape: (2, 24)
M0 mechanism shape: (2, 216)
Alpha_s shape: (24, 216)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387506, 0.1022729386438754, 0.10227293864387489, 0.047747995033239864, 0.04774799503324011, 0.047747995033240405, 0.04670317536074194, 0.04670317536074194, 0.04670317536074109, 0.058866603128812464, 0.058866603128811666, 0.05886660312881117, 0.04156271762404372, 0.04156271762404372, 0.04156271762404307, 0.030013746023441812, 0.03001374602344372, 0.030013746023442794, 0.10227293864387524, 0.10227293864387534, 0.10227293864387506, 0.04774799503324011, 0.04774799503324063, 0.04774799503324063, 0.046703175360741055, 0.04670317536074194, 0.04670317536074135, 0.05886660312881202, 0.058866603128812256, 0.05886660312881193, 0.04156271762404372, 0.04156271762404372, 0.04156271762404307, 0.03001374602344282, 0.03001374602344372, 0.030013746023442273, 0.10227293864387506, 0.10227293864387496, 0.10227293864387506, 0.04774799503323953, 0.04774799503324063, 0.04774799503323957, 0.04670317536074049, 0.046703175360741686, 0.046703175360741055, 0.05886660312881184, 0.058866603128812256, 0.058866603128812256, 0.04156271762404372, 0.041562717624043406, 0.04156271762404372, 0.030013746023442273, 0.030013746023443054, 0.030013746023442273, 0.10227293864387496, 0.10227293864387496, 0.10227293864387506, 0.04774799503323957, 0.04774799503324063, 0.04774799503324008, 0.04670317536074135, 0.046703175360741124, 0.04670317536074079, 0.05886660312881202, 0.05886660312881202, 0.058866603128812256, 0.041562717624043385, 0.04156271762404307, 0.04156271762404372, 0.030013746023442246, 0.030013746023441812, 0.030013746023442218, 0.10227293864387496, 0.10227293864387496, 0.10227293864387503, 0.047747995033240405, 0.047747995033240336, 0.04774799503324011, 0.046703175360741055, 0.046703175360741055, 0.046703175360741644, 0.058866603128812076, 0.05886660312881117, 0.058866603128812464, 0.04156271762404307, 0.04156271762404372, 0.04156271762404405, 0.030013746023442794, 0.030013746023442218, 0.030013746023442794, 0.10227293864387506, 0.10227293864387513, 0.10227293864387496, 0.04774799503324015, 0.04774799503324008, 0.04774799503323982, 0.04670317536074194, 0.046703175360741644, 0.04670317536074135, 0.05886660312881184, 0.05886660312881155, 0.05886660312881184, 0.04156271762404405, 0.04156271762404372, 0.04156271762404372, 0.030013746023442218, 0.03001374602344372, 0.03001374602344372, 0.10227293864387496, 0.1022729386438754, 0.10227293864387506, 0.04774799503323957, 0.04774799503324063, 0.047747995033240405, 0.046703175360741055, 0.046703175360741055, 0.046703175360741055, 0.058866603128812194, 0.05886660312881202, 0.05886660312881117, 0.04156271762404372, 0.04156271762404372, 0.041562717624043365, 0.03001374602344372, 0.030013746023442794, 0.030013746023443286, 0.10227293864387496, 0.10227293864387531, 0.10227293864387506, 0.04774799503323957, 0.04774799503324011, 0.04774799503324008, 0.04670317536074135, 0.04670317536074194, 0.046703175360741464, 0.058866603128812076, 0.05886660312881117, 0.0588666031288119, 0.0415627176240428, 0.0415627176240428, 0.041562717624043406, 0.030013746023441784, 0.030013746023443054, 0.030013746023442246, 0.10227293864387486, 0.10227293864387496, 0.10227293864387524, 0.047747995033240405, 0.047747995033240405, 0.04774799503324011, 0.04670317536074194, 0.04670317536074194, 0.04670317536074072, 0.058866603128811666, 0.05886660312881202, 0.05886660312881117, 0.04156271762404297, 0.04156271762404307, 0.04156271762404372, 0.03001374602344282, 0.030013746023441784, 0.030013746023442707, 0.10227293864387496, 0.10227293864387489, 0.10227293864387506, 0.047747995033240516, 0.04774799503324011, 0.047747995033239676, 0.046703175360741055, 0.046703175360741575, 0.046703175360741644, 0.05886660312881243, 0.05886660312881117, 0.058866603128812194, 0.041562717624042636, 0.04156271762404372, 0.04156271762404403, 0.030013746023442707, 0.030013746023441812, 0.03001374602344282, 0.10227293864387473, 0.10227293864387486, 0.10227293864387503, 0.047747995033239864, 0.04774799503323946, 0.04774799503324011, 0.04670317536074194, 0.04670317536074135, 0.04670317536074042, 0.05886660312881184, 0.058866603128812464, 0.058866603128812076, 0.04156271762404376, 0.041562717624043406, 0.04156271762404299, 0.030013746023442218, 0.030013746023442218, 0.030013746023441812, 0.10227293864387473, 0.10227293864387496, 0.10227293864387496, 0.04774799503324011, 0.047747995033240405, 0.04774799503324008, 0.04670317536074042, 0.04670317536074135, 0.04670317536074079, 0.05886660312881184, 0.05886660312881184, 0.05886660312881117, 0.0415627176240437, 0.04156271762404372, 0.04156271762404372, 0.03001374602344372, 0.03001374602344282, 0.03001374602344372]

Abstraction error: 0.1022729386438754

M1: ['X', 'Y', 'W'] -> ['Z']
M0: ['A', 'B', 'C', 'F'] -> ['D']
M1 mechanism shape: (2, 24)
M0 mechanism shape: (3, 144)
Alpha_s shape: (24, 144)
Alpha_t shape: (2, 3)
All JS distances: [0.02578960263558817, 0.02578960263558817, 0.019287607836718425, 0.01928760783671703, 0.1231088404272969, 0.1231088404272969, 0.07219191193089762, 0.07219191193089779, 0.055941248263735054, 0.055941248263735206, 0.014661392655724529, 0.01466139265572364, 0.09004690651961542, 0.09004690651961576, 0.04826356728265522, 0.048263567282654754, 0.09547683896743694, 0.09547683896743653, 0.11540183227374479, 0.11540183227374429, 0.07826351680322856, 0.07826351680322852, 0.10131564510398727, 0.10131564510398752, 0.05360165792477714, 0.05360165792477692, 0.02941927906019915, 0.029419279060199505, 0.03941675236485358, 0.03941675236485448, 0.06259599591646466, 0.06259599591646421, 0.04563907082185634, 0.045639070821856906, 0.07043791793351202, 0.07043791793351266, 0.05139084057708295, 0.05139084057708265, 0.022870742231004944, 0.022870742231003653, 0.001237585378362012, 0.001237585378362034, 0.041848909725623315, 0.041848909725622566, 0.019424630254393963, 0.019424630254394608, 0.0546040019713811, 0.05460400197138135, 0.05599715696418189, 0.05599715696418205, 0.027122467588032502, 0.02712246758803477, 0.020005636372829108, 0.020005636372832033, 0.054422779984729, 0.05442277998472941, 0.031810856733686194, 0.031810856733686194, 0.0632803936805915, 0.06328039368059196, 0.05544514708879597, 0.05544514708879597, 0.03213037716307666, 0.032130377163077144, 0.05372437885798039, 0.05372437885798094, 0.07100067370421372, 0.07100067370421406, 0.055219736073434264, 0.055219736073433605, 0.07655233188728923, 0.07655233188728881, 0.07028672642965776, 0.07028672642965811, 0.03980667043736613, 0.03980667043736613, 0.07636409101905714, 0.07636409101905742, 0.09236313112750635, 0.09236313112750583, 0.06840199705104488, 0.06840199705104537, 0.08907739020327914, 0.08907739020327872, 0.01519831469291298, 0.015198314692912924, 0.0001532252501715335, 0.00015322525008850256, 0.0760752507351054, 0.07607525073510556, 0.02126294205782402, 0.021262942057825182, 0.02934757578698006, 0.02934757578697965, 0.014404367914741941, 0.014404367914743779, 0.045947246941627475, 0.045947246941628266, 0.02144680588582343, 0.02144680588582454, 0.0022106708412391687, 0.002210670841233995, 0.03871882685939977, 0.03871882685940107, 0.020932998441777265, 0.02093299844177826, 0.05366155424980678, 0.053661554249806453, 0.041557830291206944, 0.041557830291207756, 0.022076302224551172, 0.022076302224552015, 0.014443698689198294, 0.014443698689196822, 0.0419614662685698, 0.04196146626856902, 0.030145187245636532, 0.03014518724563515, 0.057499037660541416, 0.05749903766054187, 0.04074699969646068, 0.04074699969646144, 0.019581160259392345, 0.01958116025939363, 6.842138417612676e-05, 6.842138434463858e-05, 0.03410170063702716, 0.03410170063702718, 0.020133384916813008, 0.020133384916813678, 0.05146466031333257, 0.05146466031333257, 0.03357601382296405, 0.03357601382296452, 0.022633868883736682, 0.02263386888373722, 0.12984449939636644, 0.12984449939636664, 0.08079336657139816, 0.08079336657139835, 0.05941471955776575, 0.05941471955776575, 0.01928271216369222, 0.01928271216369359]

Abstraction error: 0.12984449939636664

M1: ['X', 'Z', 'W'] -> ['Y']
M0: ['A', 'B', 'D', 'F'] -> ['C']
M1 mechanism shape: (4, 12)
M0 mechanism shape: (6, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307946, 0.1485260632930798, 0.14852606329307974, 0.14852606329307969, 0.1485260632930798, 0.14852606329307946, 0.09265621144301282, 0.09265621144301286, 0.09265621144301267, 0.09265621144301288, 0.09265621144301284, 0.09265621144301305, 0.12001084898732209, 0.12001084898732223, 0.12001084898732194, 0.12001084898732232, 0.12001084898732234, 0.12001084898732232, 0.15997989391107822, 0.15997989391107814, 0.15997989391107836, 0.15997989391107853, 0.15997989391107834, 0.1599798939110782, 0.16203192810476505, 0.1620319281047651, 0.1620319281047651, 0.1620319281047652, 0.16203192810476527, 0.1620319281047651, 0.1269668098076927, 0.12696680980769232, 0.12696680980769245, 0.1269668098076926, 0.12696680980769265, 0.1269668098076925, 0.10069809180846702, 0.10069809180846702, 0.10069809180846678, 0.10069809180846685, 0.10069809180846676, 0.10069809180846669, 0.18636973290336367, 0.1863697329033637, 0.18636973290336384, 0.1863697329033637, 0.18636973290336384, 0.18636973290336384, 0.10476951328179755, 0.10476951328179754, 0.10476951328179747, 0.10476951328179744, 0.10476951328179754, 0.10476951328179743, 0.11807985631483503, 0.1180798563148352, 0.11807985631483502, 0.1180798563148349, 0.11807985631483509, 0.11807985631483503, 0.11792531349738833, 0.11792531349738843, 0.11792531349738837, 0.11792531349738845, 0.1179253134973884, 0.11792531349738833, 0.18437104992516803, 0.18437104992516815, 0.18437104992516803, 0.18437104992516815, 0.18437104992516798, 0.18437104992516798]

Abstraction error: 0.18636973290336384


OVERALL ABSTRACTION ERROR: 0.22883249205446551

Automating the evaluation of abstraction error¶

To finalize the process of automation we package and move our code inside the class Abstraction in src/pgm_models.py. We define a function evaluate_abstraction_error() which, besides a verbosity parameter, takes two key parameters:

metric: a function defining how to compute the distance among interventional distribution. If no function is passed, the algorithm falls back on JSD.
J_algorithm: a function used to compute the evaluation set $\mathcal{J}$. If no function is passed, the algorithms falls back on the last algorithm we defined in this notebook (enumerating all pairs for which there is a directed path in the base or in the abstracted model).

These two parameters are key component for evaluating abstraction error, determining the quality and the computational cost of the process. In particular, the J_algorithm determines the cardinality of the set of pairs to evaluate and therefore has a direct impact on the complexity of the algorithm. Heuristics may be used to reduce this set and allow an efficient computation.

Example 5¶

We now look at a last, even more complex abstract case, and use our Abstraction object.

Model definition¶

We will use the exact same model we implemented above.

Running the computational loop¶

In [38]:

result = A.evaluate_abstraction_error(verbose=True)

- Checking ['X'] -> ['Y']: True
- Checking ['X'] -> ['Z']: True
- Checking ['X'] -> ['W']: True
- Checking ['X'] -> ['Y', 'Z']: True
- Checking ['X'] -> ['Y', 'W']: True
- Checking ['X'] -> ['Z', 'W']: True
- Checking ['X'] -> ['Y', 'Z', 'W']: True
- Checking ['Y'] -> ['X']: False
---- Checking ['C'] -> ['A', 'B']: False
- Checking ['Y'] -> ['Z']: True
- Checking ['Y'] -> ['W']: True
- Checking ['Y'] -> ['X', 'Z']: True
- Checking ['Y'] -> ['X', 'W']: True
- Checking ['Y'] -> ['Z', 'W']: True
- Checking ['Y'] -> ['X', 'Z', 'W']: True
- Checking ['Z'] -> ['X']: False
---- Checking ['D'] -> ['A', 'B']: False
- Checking ['Z'] -> ['Y']: False
---- Checking ['D'] -> ['C']: False
- Checking ['Z'] -> ['W']: False
---- Checking ['D'] -> ['F']: False
- Checking ['Z'] -> ['X', 'Y']: False
---- Checking ['D'] -> ['A', 'B', 'C']: False
- Checking ['Z'] -> ['X', 'W']: False
---- Checking ['D'] -> ['A', 'B', 'F']: False
- Checking ['Z'] -> ['Y', 'W']: False
---- Checking ['D'] -> ['C', 'F']: False
- Checking ['Z'] -> ['X', 'Y', 'W']: False
---- Checking ['D'] -> ['A', 'B', 'C', 'F']: False
- Checking ['W'] -> ['X']: False
---- Checking ['F'] -> ['A', 'B']: False
- Checking ['W'] -> ['Y']: False
---- Checking ['F'] -> ['C']: False
- Checking ['W'] -> ['Z']: False
---- Checking ['F'] -> ['D']: False
- Checking ['W'] -> ['X', 'Y']: False
---- Checking ['F'] -> ['A', 'B', 'C']: False
- Checking ['W'] -> ['X', 'Z']: False
---- Checking ['F'] -> ['A', 'B', 'D']: False
- Checking ['W'] -> ['Y', 'Z']: False
---- Checking ['F'] -> ['C', 'D']: False
- Checking ['W'] -> ['X', 'Y', 'Z']: False
---- Checking ['F'] -> ['A', 'B', 'C', 'D']: False
- Checking ['X', 'Y'] -> ['Z']: True
- Checking ['X', 'Y'] -> ['W']: True
- Checking ['X', 'Y'] -> ['Z', 'W']: True
- Checking ['X', 'Z'] -> ['Y']: True
- Checking ['X', 'Z'] -> ['W']: True
- Checking ['X', 'Z'] -> ['Y', 'W']: True
- Checking ['X', 'W'] -> ['Y']: True
- Checking ['X', 'W'] -> ['Z']: True
- Checking ['X', 'W'] -> ['Y', 'Z']: True
- Checking ['Y', 'Z'] -> ['X']: False
---- Checking ['C', 'D'] -> ['A', 'B']: False
- Checking ['Y', 'Z'] -> ['W']: True
- Checking ['Y', 'Z'] -> ['X', 'W']: True
- Checking ['Y', 'W'] -> ['X']: False
---- Checking ['C', 'F'] -> ['A', 'B']: False
- Checking ['Y', 'W'] -> ['Z']: True
- Checking ['Y', 'W'] -> ['X', 'Z']: True
- Checking ['Z', 'W'] -> ['X']: False
---- Checking ['D', 'F'] -> ['A', 'B']: False
- Checking ['Z', 'W'] -> ['Y']: False
---- Checking ['D', 'F'] -> ['C']: False
- Checking ['Z', 'W'] -> ['X', 'Y']: False
---- Checking ['D', 'F'] -> ['A', 'B', 'C']: False
- Checking ['X', 'Y', 'Z'] -> ['W']: True
- Checking ['X', 'Y', 'W'] -> ['Z']: True
- Checking ['X', 'Z', 'W'] -> ['Y']: True
- Checking ['Y', 'Z', 'W'] -> ['X']: False
---- Checking ['C', 'D', 'F'] -> ['A', 'B']: False

 29 legitimate pairs of sets out of 225 possbile pairs of sets

M1: ['X'] -> ['Y']
M0: ['A', 'B'] -> ['C']
M1 mechanism shape: (4, 3)
M0 mechanism shape: (6, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307985, 0.09265621144301286, 0.12001084898732219, 0.15997989391107853, 0.16203192810476508, 0.12696680980769243, 0.10069809180846691, 0.1863697329033639, 0.1047695132817974, 0.11807985631483506, 0.1179253134973884, 0.18437104992516792]

Abstraction error: 0.1863697329033639

M1: ['X'] -> ['Z']
M0: ['A', 'B'] -> ['D']
M1 mechanism shape: (2, 3)
M0 mechanism shape: (3, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (2, 3)
All JS distances: [0.0351304898487227, 0.06733630947989032, 0.0313707716587073, 0.0329667651770552, 0.03126112349484746, 0.03158249424275995, 0.047243008500498236, 0.0019017729573616153, 0.03424179787827428, 0.02209397224515851, 0.023474779635804806, 0.04781008084384352]

Abstraction error: 0.06733630947989032

M1: ['X'] -> ['W']
M0: ['A', 'B'] -> ['F']
M1 mechanism shape: (2, 3)
M0 mechanism shape: (2, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (2, 2)
All JS distances: [0.030672034914896472, 0.045551698121225746, 0.047482514947993826, 0.040984523781884684, 0.041699048049716914, 0.04609574520099206, 0.04539146222920609, 0.04707914958025027, 0.04617497146167864, 0.041023811525147724, 0.044405940952980685, 0.04921460205222507]

Abstraction error: 0.04921460205222507

M1: ['X'] -> ['Y', 'Z']
M0: ['A', 'B'] -> ['C', 'D']
M1 mechanism shape: (8, 3)
M0 mechanism shape: (18, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (8, 18)
All JS distances: [0.18714930747588385, 0.16342782470143777, 0.13923688032004672, 0.16996752417713565, 0.17244633676294255, 0.15413332692010762, 0.15329859180094013, 0.20291390546039728, 0.11960094914967821, 0.12933417631167846, 0.12854231172535505, 0.22176385875401214]

Abstraction error: 0.22176385875401214

M1: ['X'] -> ['Y', 'W']
M0: ['A', 'B'] -> ['C', 'F']
M1 mechanism shape: (8, 3)
M0 mechanism shape: (12, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (8, 12)
All JS distances: [0.15674163338369104, 0.11027827540763376, 0.1344765032870174, 0.16900387860096566, 0.17132352601049652, 0.13971800215379107, 0.11679126446490122, 0.19539161442730005, 0.12050680001965054, 0.13092227531421416, 0.13112660874946497, 0.19297402947863096]

Abstraction error: 0.19539161442730005

M1: ['X'] -> ['Z', 'W']
M0: ['A', 'B'] -> ['D', 'F']
M1 mechanism shape: (4, 3)
M0 mechanism shape: (6, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (4, 6)
All JS distances: [0.047192705325690115, 0.08127081190113358, 0.05706325256989926, 0.0529727107172862, 0.052399220810823675, 0.055975864083481014, 0.06555189636445083, 0.047829683686611724, 0.05780309803964864, 0.04684660609816745, 0.05055104512955366, 0.06868965197737846]

Abstraction error: 0.08127081190113358

M1: ['X'] -> ['Y', 'Z', 'W']
M0: ['A', 'B'] -> ['C', 'D', 'F']
M1 mechanism shape: (16, 3)
M0 mechanism shape: (36, 12)
Alpha_s shape: (3, 12)
Alpha_t shape: (16, 36)
All JS distances: [0.1936393333585678, 0.17380671152417954, 0.1517981935295362, 0.178458587800174, 0.1811617242150942, 0.16470162425368473, 0.16415818702211926, 0.2111959555128114, 0.1335662377117007, 0.14111186401600812, 0.1407190157079215, 0.22883249205446551]

Abstraction error: 0.22883249205446551

M1: ['Y'] -> ['Z']
M0: ['C'] -> ['D']
M1 mechanism shape: (2, 4)
M0 mechanism shape: (3, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (2, 3)
All JS distances: [0.06099916587205272, 0.032682775053057354, 0.04679729540729152, 0.07128377077244254, 0.049598194647336426, 0.07518600631595818]

Abstraction error: 0.07518600631595818

M1: ['Y'] -> ['W']
M0: ['C'] -> ['F']
M1 mechanism shape: (2, 4)
M0 mechanism shape: (2, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387506, 0.047747995033239246, 0.04670317536074079, 0.05886660312881184, 0.04156271762404307, 0.030013746023442218]

Abstraction error: 0.10227293864387506

M1: ['Y'] -> ['X', 'Z']
M0: ['C'] -> ['A', 'B', 'D']
M1 mechanism shape: (6, 4)
M0 mechanism shape: (36, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (6, 36)
All JS distances: [0.17389072631509284, 0.14619836813152523, 0.19798973566307185, 0.19883946724488175, 0.16388446559385356, 0.16929406716985493]

Abstraction error: 0.19883946724488175

M1: ['Y'] -> ['X', 'W']
M0: ['C'] -> ['A', 'B', 'F']
M1 mechanism shape: (6, 4)
M0 mechanism shape: (24, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (6, 24)
All JS distances: [0.1688062382679865, 0.14357083895353628, 0.14323935375592675, 0.14749571686086949, 0.141704732098744, 0.13886536590930515]

Abstraction error: 0.1688062382679865

M1: ['Y'] -> ['Z', 'W']
M0: ['C'] -> ['D', 'F']
M1 mechanism shape: (4, 4)
M0 mechanism shape: (6, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (4, 6)
All JS distances: [0.1187091362816008, 0.05814390050461374, 0.06633832145879023, 0.09232883649684216, 0.06498429821122392, 0.08105295591377457]

Abstraction error: 0.1187091362816008

M1: ['Y'] -> ['X', 'Z', 'W']
M0: ['C'] -> ['A', 'B', 'D', 'F']
M1 mechanism shape: (12, 4)
M0 mechanism shape: (72, 6)
Alpha_s shape: (4, 6)
Alpha_t shape: (12, 72)
All JS distances: [0.19992799783427462, 0.15363108648024623, 0.2032521275291327, 0.20662976317432394, 0.16893301270801445, 0.17187442878295403]

Abstraction error: 0.20662976317432394

M1: ['X', 'Y'] -> ['Z']
M0: ['A', 'B', 'C'] -> ['D']
M1 mechanism shape: (2, 12)
M0 mechanism shape: (3, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (2, 3)
All JS distances: [0.025789602635587733, 0.01928760783671703, 0.12310884042729682, 0.07219191193089779, 0.05594124826373502, 0.01466139265572651, 0.09004690651961561, 0.04826356728265454, 0.09547683896743701, 0.11540183227374494, 0.07826351680322807, 0.10131564510398797, 0.053601657924777175, 0.02941927906020033, 0.03941675236485532, 0.06259599591646454, 0.045639070821855615, 0.07043791793351223, 0.05139084057708265, 0.022870742231003844, 0.001237585378362012, 0.041848909725622566, 0.019424630254393963, 0.05460400197138135, 0.05599715696418146, 0.027122467588032502, 0.020005636372832054, 0.05442277998472925, 0.031810856733685486, 0.06328039368059145, 0.055445147088796, 0.03213037716307588, 0.05372437885798094, 0.07100067370421342, 0.055219736073434514, 0.07655233188728902, 0.07028672642965786, 0.03980667043736547, 0.07636409101905753, 0.09236313112750567, 0.06840199705104537, 0.08907739020327865, 0.01519831469291378, 0.00015322525000538317, 0.07607525073510538, 0.021262942057825262, 0.029347575786980565, 0.014404367914743779, 0.04594724694162721, 0.02144680588582337, 0.0022106708412245294, 0.03871882685939943, 0.02093299844177826, 0.053661554249806453, 0.041557830291206355, 0.022076302224551918, 0.014443698689199013, 0.0419614662685693, 0.0301451872456355, 0.05749903766054175, 0.04074699969646032, 0.019581160259392962, 6.842138417617628e-05, 0.03410170063702787, 0.020133384916813008, 0.0514646603133323, 0.033576013822963664, 0.022633868883736682, 0.12984449939636644, 0.08079336657139817, 0.059414719557766074, 0.019282712163695793]

Abstraction error: 0.12984449939636644

M1: ['X', 'Y'] -> ['W']
M0: ['A', 'B', 'C'] -> ['F']
M1 mechanism shape: (2, 12)
M0 mechanism shape: (2, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387503, 0.04774799503323982, 0.04670317536074194, 0.058866603128811666, 0.041562717624043385, 0.03001374602344268, 0.10227293864387496, 0.04774799503323982, 0.046703175360741464, 0.0588666031288114, 0.041562717624043385, 0.030013746023443286, 0.10227293864387489, 0.047747995033239864, 0.046703175360741055, 0.05886660312881184, 0.04156271762404307, 0.030013746023441812, 0.102272938643875, 0.047747995033240405, 0.04670317536074079, 0.05886660312881117, 0.041562717624043385, 0.030013746023442273, 0.10227293864387506, 0.04774799503324011, 0.046703175360741055, 0.05886660312881164, 0.04156271762404307, 0.030013746023441784, 0.10227293864387473, 0.047747995033240405, 0.04670317536074109, 0.05886660312881117, 0.041562717624043385, 0.03001374602344372, 0.10227293864387489, 0.04774799503323953, 0.04670317536074172, 0.05886660312881202, 0.04156271762404372, 0.03001374602344184, 0.102272938643875, 0.047747995033239864, 0.04670317536074079, 0.05886660312881214, 0.041562717624043385, 0.030013746023442218, 0.10227293864387496, 0.04774799503323917, 0.04670317536074135, 0.058866603128812256, 0.041562717624043385, 0.030013746023442794, 0.10227293864387534, 0.04774799503323975, 0.04670317536074049, 0.058866603128811666, 0.041562717624043385, 0.030013746023442273, 0.1022729386438754, 0.04774799503324008, 0.046703175360741644, 0.058866603128811666, 0.041562717624042386, 0.030013746023443286, 0.10227293864387506, 0.047747995033240336, 0.04670317536074194, 0.05886660312881193, 0.041562717624043385, 0.030013746023443113]

Abstraction error: 0.1022729386438754

M1: ['X', 'Y'] -> ['Z', 'W']
M0: ['A', 'B', 'C'] -> ['D', 'F']
M1 mechanism shape: (4, 12)
M0 mechanism shape: (6, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (4, 6)
All JS distances: [0.1056624636155618, 0.052084731108216153, 0.1314612858388807, 0.09276636503927882, 0.06956464778069847, 0.03438890194347461, 0.13531096368386336, 0.06803935493658374, 0.10652381872777678, 0.12915427868645055, 0.08881093554411168, 0.1056417634958173, 0.11519772939245851, 0.056367214614913355, 0.06134614046012533, 0.08585858786558867, 0.062008491375707214, 0.07667885627900818, 0.11445924091758652, 0.05348776375112503, 0.046759883610066603, 0.07228635207813679, 0.0462191596341699, 0.0626470708452943, 0.11643580818005946, 0.0553474006039522, 0.050928365869165666, 0.08015841444284152, 0.05266507220301004, 0.07026312381719627, 0.11597516645244803, 0.05776071058397569, 0.07147565036567279, 0.0921053837481447, 0.0693613171125721, 0.08228007437489032, 0.12347415890694341, 0.06232328459792683, 0.0898023235241765, 0.10927070948737225, 0.08025302982041745, 0.09399854331758245, 0.10395604804843733, 0.04856581683701331, 0.08914721232055513, 0.06260268068822607, 0.05093508167652123, 0.03433159933718795, 0.112132560182083, 0.052834658560287, 0.04681448119141199, 0.07052868157394003, 0.04687061220448419, 0.06179520612717915, 0.11034076169497048, 0.052972029514956304, 0.049037825077057624, 0.07233741738004239, 0.05165774297906071, 0.06507406711816113, 0.11013504899786426, 0.05207268084117108, 0.04676312167240142, 0.06811731337283324, 0.04651110833275453, 0.05988487637176444, 0.10768177942575126, 0.05336316759911026, 0.13775997628096487, 0.0995174522023532, 0.07236330825668591, 0.0365645649668503]

Abstraction error: 0.13775997628096487

M1: ['X', 'Z'] -> ['Y']
M0: ['A', 'B', 'D'] -> ['C']
M1 mechanism shape: (4, 6)
M0 mechanism shape: (6, 36)
Alpha_s shape: (6, 36)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307957, 0.1485260632930798, 0.14852606329307955, 0.09265621144301266, 0.09265621144301273, 0.0926562114430127, 0.12001084898732221, 0.12001084898732214, 0.12001084898732231, 0.1599798939110785, 0.15997989391107822, 0.15997989391107834, 0.1620319281047652, 0.16203192810476508, 0.16203192810476508, 0.12696680980769237, 0.1269668098076927, 0.12696680980769257, 0.10069809180846705, 0.10069809180846673, 0.10069809180846714, 0.18636973290336362, 0.18636973290336362, 0.1863697329033639, 0.1047695132817973, 0.10476951328179758, 0.10476951328179743, 0.11807985631483495, 0.11807985631483514, 0.11807985631483495, 0.11792531349738826, 0.11792531349738844, 0.11792531349738834, 0.18437104992516795, 0.18437104992516803, 0.1843710499251681]

Abstraction error: 0.1863697329033639

M1: ['X', 'Z'] -> ['W']
M0: ['A', 'B', 'D'] -> ['F']
M1 mechanism shape: (2, 6)
M0 mechanism shape: (2, 36)
Alpha_s shape: (6, 36)
Alpha_t shape: (2, 2)
All JS distances: [0.03067203491489729, 0.030672034914896472, 0.030672034914896895, 0.045551698121225746, 0.04555169812122603, 0.045551698121225746, 0.04748251494799353, 0.047482514947992945, 0.047482514947992945, 0.0409845237818843, 0.040984523781884684, 0.04098452378188504, 0.041699048049717226, 0.041699048049717226, 0.041699048049717205, 0.04609574520099206, 0.04609574520099146, 0.04609574520099236, 0.04539146222920569, 0.04539146222920508, 0.045391462229205155, 0.04707914958024961, 0.04707914958024957, 0.04707914958024961, 0.04617497146167895, 0.04617497146167951, 0.04617497146167917, 0.04102381152514819, 0.04102381152514891, 0.041023811525147516, 0.04440594095298096, 0.044405940952980685, 0.044405940952980984, 0.04921460205222507, 0.04921460205222507, 0.049214602052225276]

Abstraction error: 0.049214602052225276

M1: ['X', 'Z'] -> ['Y', 'W']
M0: ['A', 'B', 'D'] -> ['C', 'F']
M1 mechanism shape: (8, 6)
M0 mechanism shape: (12, 36)
Alpha_s shape: (6, 36)
Alpha_t shape: (8, 12)
All JS distances: [0.15674163338369085, 0.15674163338369088, 0.15674163338369085, 0.11027827540763371, 0.11027827540763364, 0.11027827540763355, 0.13447650328701752, 0.13447650328701752, 0.1344765032870175, 0.16900387860096577, 0.16900387860096575, 0.16900387860096572, 0.1713235260104964, 0.17132352601049647, 0.17132352601049639, 0.13971800215379096, 0.13971800215379113, 0.13971800215379096, 0.1167912644649011, 0.11679126446490126, 0.11679126446490122, 0.19539161442730002, 0.19539161442730005, 0.19539161442729996, 0.12050680001965042, 0.1205068000196504, 0.12050680001965045, 0.13092227531421435, 0.13092227531421438, 0.13092227531421424, 0.13112660874946505, 0.131126608749465, 0.13112660874946483, 0.19297402947863102, 0.19297402947863096, 0.19297402947863096]

Abstraction error: 0.19539161442730005

M1: ['X', 'W'] -> ['Y']
M0: ['A', 'B', 'F'] -> ['C']
M1 mechanism shape: (4, 6)
M0 mechanism shape: (6, 24)
Alpha_s shape: (6, 24)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307963, 0.14852606329307957, 0.09265621144301267, 0.09265621144301286, 0.12001084898732224, 0.12001084898732221, 0.15997989391107834, 0.15997989391107845, 0.16203192810476497, 0.16203192810476522, 0.12696680980769276, 0.12696680980769245, 0.10069809180846709, 0.10069809180846699, 0.18636973290336376, 0.1863697329033638, 0.10476951328179737, 0.10476951328179754, 0.11807985631483475, 0.11807985631483489, 0.11792531349738826, 0.11792531349738844, 0.18437104992516803, 0.18437104992516806]

Abstraction error: 0.1863697329033638

M1: ['X', 'W'] -> ['Z']
M0: ['A', 'B', 'F'] -> ['D']
M1 mechanism shape: (2, 6)
M0 mechanism shape: (3, 24)
Alpha_s shape: (6, 24)
Alpha_t shape: (2, 3)
All JS distances: [0.035130489848721955, 0.03513048984872272, 0.06733630947989032, 0.06733630947989032, 0.03137077165870871, 0.03137077165870871, 0.03296676517705488, 0.03296676517705488, 0.03126112349484666, 0.03126112349484746, 0.03158249424276041, 0.03158249424276083, 0.04724300850049828, 0.047243008500497945, 0.0019017729573485886, 0.0019017729573485886, 0.034241797878275367, 0.03424179787827465, 0.02209397224515739, 0.02209397224515739, 0.023474779635806173, 0.023474779635806173, 0.04781008084384352, 0.04781008084384316]

Abstraction error: 0.06733630947989032

M1: ['X', 'W'] -> ['Y', 'Z']
M0: ['A', 'B', 'F'] -> ['C', 'D']
M1 mechanism shape: (8, 6)
M0 mechanism shape: (18, 24)
Alpha_s shape: (6, 24)
Alpha_t shape: (8, 18)
All JS distances: [0.18714930747588385, 0.18714930747588387, 0.1634278247014377, 0.16342782470143763, 0.13923688032004664, 0.1392368803200467, 0.16996752417713568, 0.1699675241771357, 0.17244633676294252, 0.17244633676294258, 0.15413332692010767, 0.15413332692010773, 0.15329859180094013, 0.15329859180094002, 0.20291390546039717, 0.20291390546039711, 0.11960094914967835, 0.11960094914967805, 0.12933417631167843, 0.12933417631167834, 0.1285423117253549, 0.1285423117253551, 0.22176385875401208, 0.22176385875401208]

Abstraction error: 0.22176385875401208

M1: ['Y', 'Z'] -> ['W']
M0: ['C', 'D'] -> ['F']
M1 mechanism shape: (2, 8)
M0 mechanism shape: (2, 18)
Alpha_s shape: (8, 18)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387506, 0.10227293864387496, 0.10227293864387489, 0.047747995033239246, 0.04774799503324015, 0.04774799503323957, 0.04670317536074049, 0.04670317536074139, 0.046703175360741464, 0.05886660312881117, 0.05886660312881202, 0.05886660312881202, 0.04156271762404405, 0.04156271762404405, 0.04156271762404372, 0.030013746023442218, 0.030013746023441784, 0.030013746023442218]

Abstraction error: 0.10227293864387506

M1: ['Y', 'Z'] -> ['X', 'W']
M0: ['C', 'D'] -> ['A', 'B', 'F']
M1 mechanism shape: (6, 8)
M0 mechanism shape: (24, 18)
Alpha_s shape: (8, 18)
Alpha_t shape: (6, 24)
All JS distances: [0.1688062382679865, 0.1688062382679864, 0.16880623826798657, 0.14357083895353626, 0.14357083895353628, 0.14357083895353615, 0.1432393537559268, 0.14323935375592697, 0.14323935375592672, 0.14749571686086962, 0.14749571686086962, 0.1474957168608697, 0.14170473209874385, 0.14170473209874374, 0.14170473209874376, 0.13886536590930534, 0.1388653659093051, 0.138865365909305]

Abstraction error: 0.16880623826798657

M1: ['Y', 'W'] -> ['Z']
M0: ['C', 'F'] -> ['D']
M1 mechanism shape: (2, 8)
M0 mechanism shape: (3, 12)
Alpha_s shape: (8, 12)
Alpha_t shape: (2, 3)
All JS distances: [0.060999165872052154, 0.060999165872052154, 0.032682775053056, 0.03268277505305698, 0.046797295407291155, 0.0467972954072918, 0.07128377077244273, 0.07128377077244222, 0.04959819464733618, 0.049598194647336426, 0.07518600631595801, 0.07518600631595769]

Abstraction error: 0.07518600631595801

M1: ['Y', 'W'] -> ['X', 'Z']
M0: ['C', 'F'] -> ['A', 'B', 'D']
M1 mechanism shape: (6, 8)
M0 mechanism shape: (36, 12)
Alpha_s shape: (8, 12)
Alpha_t shape: (6, 36)
All JS distances: [0.17389072631509295, 0.17389072631509261, 0.14619836813152512, 0.1461983681315251, 0.1979897356630717, 0.19798973566307185, 0.19883946724488183, 0.19883946724488177, 0.16388446559385353, 0.16388446559385347, 0.16929406716985484, 0.16929406716985496]

Abstraction error: 0.19883946724488183

M1: ['X', 'Y', 'Z'] -> ['W']
M0: ['A', 'B', 'C', 'D'] -> ['F']
M1 mechanism shape: (2, 24)
M0 mechanism shape: (2, 216)
Alpha_s shape: (24, 216)
Alpha_t shape: (2, 2)
All JS distances: [0.10227293864387506, 0.1022729386438754, 0.10227293864387489, 0.047747995033239864, 0.04774799503324011, 0.047747995033240405, 0.04670317536074194, 0.04670317536074194, 0.04670317536074109, 0.058866603128812464, 0.058866603128811666, 0.05886660312881117, 0.04156271762404372, 0.04156271762404372, 0.04156271762404307, 0.030013746023441812, 0.03001374602344372, 0.030013746023442794, 0.10227293864387524, 0.10227293864387534, 0.10227293864387506, 0.04774799503324011, 0.04774799503324063, 0.04774799503324063, 0.046703175360741055, 0.04670317536074194, 0.04670317536074135, 0.05886660312881202, 0.058866603128812256, 0.05886660312881193, 0.04156271762404372, 0.04156271762404372, 0.04156271762404307, 0.03001374602344282, 0.03001374602344372, 0.030013746023442273, 0.10227293864387506, 0.10227293864387496, 0.10227293864387506, 0.04774799503323953, 0.04774799503324063, 0.04774799503323957, 0.04670317536074049, 0.046703175360741686, 0.046703175360741055, 0.05886660312881184, 0.058866603128812256, 0.058866603128812256, 0.04156271762404372, 0.041562717624043406, 0.04156271762404372, 0.030013746023442273, 0.030013746023443054, 0.030013746023442273, 0.10227293864387496, 0.10227293864387496, 0.10227293864387506, 0.04774799503323957, 0.04774799503324063, 0.04774799503324008, 0.04670317536074135, 0.046703175360741124, 0.04670317536074079, 0.05886660312881202, 0.05886660312881202, 0.058866603128812256, 0.041562717624043385, 0.04156271762404307, 0.04156271762404372, 0.030013746023442246, 0.030013746023441812, 0.030013746023442218, 0.10227293864387496, 0.10227293864387496, 0.10227293864387503, 0.047747995033240405, 0.047747995033240336, 0.04774799503324011, 0.046703175360741055, 0.046703175360741055, 0.046703175360741644, 0.058866603128812076, 0.05886660312881117, 0.058866603128812464, 0.04156271762404307, 0.04156271762404372, 0.04156271762404405, 0.030013746023442794, 0.030013746023442218, 0.030013746023442794, 0.10227293864387506, 0.10227293864387513, 0.10227293864387496, 0.04774799503324015, 0.04774799503324008, 0.04774799503323982, 0.04670317536074194, 0.046703175360741644, 0.04670317536074135, 0.05886660312881184, 0.05886660312881155, 0.05886660312881184, 0.04156271762404405, 0.04156271762404372, 0.04156271762404372, 0.030013746023442218, 0.03001374602344372, 0.03001374602344372, 0.10227293864387496, 0.1022729386438754, 0.10227293864387506, 0.04774799503323957, 0.04774799503324063, 0.047747995033240405, 0.046703175360741055, 0.046703175360741055, 0.046703175360741055, 0.058866603128812194, 0.05886660312881202, 0.05886660312881117, 0.04156271762404372, 0.04156271762404372, 0.041562717624043365, 0.03001374602344372, 0.030013746023442794, 0.030013746023443286, 0.10227293864387496, 0.10227293864387531, 0.10227293864387506, 0.04774799503323957, 0.04774799503324011, 0.04774799503324008, 0.04670317536074135, 0.04670317536074194, 0.046703175360741464, 0.058866603128812076, 0.05886660312881117, 0.0588666031288119, 0.0415627176240428, 0.0415627176240428, 0.041562717624043406, 0.030013746023441784, 0.030013746023443054, 0.030013746023442246, 0.10227293864387486, 0.10227293864387496, 0.10227293864387524, 0.047747995033240405, 0.047747995033240405, 0.04774799503324011, 0.04670317536074194, 0.04670317536074194, 0.04670317536074072, 0.058866603128811666, 0.05886660312881202, 0.05886660312881117, 0.04156271762404297, 0.04156271762404307, 0.04156271762404372, 0.03001374602344282, 0.030013746023441784, 0.030013746023442707, 0.10227293864387496, 0.10227293864387489, 0.10227293864387506, 0.047747995033240516, 0.04774799503324011, 0.047747995033239676, 0.046703175360741055, 0.046703175360741575, 0.046703175360741644, 0.05886660312881243, 0.05886660312881117, 0.058866603128812194, 0.041562717624042636, 0.04156271762404372, 0.04156271762404403, 0.030013746023442707, 0.030013746023441812, 0.03001374602344282, 0.10227293864387473, 0.10227293864387486, 0.10227293864387503, 0.047747995033239864, 0.04774799503323946, 0.04774799503324011, 0.04670317536074194, 0.04670317536074135, 0.04670317536074042, 0.05886660312881184, 0.058866603128812464, 0.058866603128812076, 0.04156271762404376, 0.041562717624043406, 0.04156271762404299, 0.030013746023442218, 0.030013746023442218, 0.030013746023441812, 0.10227293864387473, 0.10227293864387496, 0.10227293864387496, 0.04774799503324011, 0.047747995033240405, 0.04774799503324008, 0.04670317536074042, 0.04670317536074135, 0.04670317536074079, 0.05886660312881184, 0.05886660312881184, 0.05886660312881117, 0.0415627176240437, 0.04156271762404372, 0.04156271762404372, 0.03001374602344372, 0.03001374602344282, 0.03001374602344372]

Abstraction error: 0.1022729386438754

M1: ['X', 'Y', 'W'] -> ['Z']
M0: ['A', 'B', 'C', 'F'] -> ['D']
M1 mechanism shape: (2, 24)
M0 mechanism shape: (3, 144)
Alpha_s shape: (24, 144)
Alpha_t shape: (2, 3)
All JS distances: [0.02578960263558817, 0.02578960263558817, 0.019287607836718425, 0.01928760783671703, 0.1231088404272969, 0.1231088404272969, 0.07219191193089762, 0.07219191193089779, 0.055941248263735054, 0.055941248263735206, 0.014661392655724529, 0.01466139265572364, 0.09004690651961542, 0.09004690651961576, 0.04826356728265522, 0.048263567282654754, 0.09547683896743694, 0.09547683896743653, 0.11540183227374479, 0.11540183227374429, 0.07826351680322856, 0.07826351680322852, 0.10131564510398727, 0.10131564510398752, 0.05360165792477714, 0.05360165792477692, 0.02941927906019915, 0.029419279060199505, 0.03941675236485358, 0.03941675236485448, 0.06259599591646466, 0.06259599591646421, 0.04563907082185634, 0.045639070821856906, 0.07043791793351202, 0.07043791793351266, 0.05139084057708295, 0.05139084057708265, 0.022870742231004944, 0.022870742231003653, 0.001237585378362012, 0.001237585378362034, 0.041848909725623315, 0.041848909725622566, 0.019424630254393963, 0.019424630254394608, 0.0546040019713811, 0.05460400197138135, 0.05599715696418189, 0.05599715696418205, 0.027122467588032502, 0.02712246758803477, 0.020005636372829108, 0.020005636372832033, 0.054422779984729, 0.05442277998472941, 0.031810856733686194, 0.031810856733686194, 0.0632803936805915, 0.06328039368059196, 0.05544514708879597, 0.05544514708879597, 0.03213037716307666, 0.032130377163077144, 0.05372437885798039, 0.05372437885798094, 0.07100067370421372, 0.07100067370421406, 0.055219736073434264, 0.055219736073433605, 0.07655233188728923, 0.07655233188728881, 0.07028672642965776, 0.07028672642965811, 0.03980667043736613, 0.03980667043736613, 0.07636409101905714, 0.07636409101905742, 0.09236313112750635, 0.09236313112750583, 0.06840199705104488, 0.06840199705104537, 0.08907739020327914, 0.08907739020327872, 0.01519831469291298, 0.015198314692912924, 0.0001532252501715335, 0.00015322525008850256, 0.0760752507351054, 0.07607525073510556, 0.02126294205782402, 0.021262942057825182, 0.02934757578698006, 0.02934757578697965, 0.014404367914741941, 0.014404367914743779, 0.045947246941627475, 0.045947246941628266, 0.02144680588582343, 0.02144680588582454, 0.0022106708412391687, 0.002210670841233995, 0.03871882685939977, 0.03871882685940107, 0.020932998441777265, 0.02093299844177826, 0.05366155424980678, 0.053661554249806453, 0.041557830291206944, 0.041557830291207756, 0.022076302224551172, 0.022076302224552015, 0.014443698689198294, 0.014443698689196822, 0.0419614662685698, 0.04196146626856902, 0.030145187245636532, 0.03014518724563515, 0.057499037660541416, 0.05749903766054187, 0.04074699969646068, 0.04074699969646144, 0.019581160259392345, 0.01958116025939363, 6.842138417612676e-05, 6.842138434463858e-05, 0.03410170063702716, 0.03410170063702718, 0.020133384916813008, 0.020133384916813678, 0.05146466031333257, 0.05146466031333257, 0.03357601382296405, 0.03357601382296452, 0.022633868883736682, 0.02263386888373722, 0.12984449939636644, 0.12984449939636664, 0.08079336657139816, 0.08079336657139835, 0.05941471955776575, 0.05941471955776575, 0.01928271216369222, 0.01928271216369359]

Abstraction error: 0.12984449939636664

M1: ['X', 'Z', 'W'] -> ['Y']
M0: ['A', 'B', 'D', 'F'] -> ['C']
M1 mechanism shape: (4, 12)
M0 mechanism shape: (6, 72)
Alpha_s shape: (12, 72)
Alpha_t shape: (4, 6)
All JS distances: [0.14852606329307946, 0.1485260632930798, 0.14852606329307974, 0.14852606329307969, 0.1485260632930798, 0.14852606329307946, 0.09265621144301282, 0.09265621144301286, 0.09265621144301267, 0.09265621144301288, 0.09265621144301284, 0.09265621144301305, 0.12001084898732209, 0.12001084898732223, 0.12001084898732194, 0.12001084898732232, 0.12001084898732234, 0.12001084898732232, 0.15997989391107822, 0.15997989391107814, 0.15997989391107836, 0.15997989391107853, 0.15997989391107834, 0.1599798939110782, 0.16203192810476505, 0.1620319281047651, 0.1620319281047651, 0.1620319281047652, 0.16203192810476527, 0.1620319281047651, 0.1269668098076927, 0.12696680980769232, 0.12696680980769245, 0.1269668098076926, 0.12696680980769265, 0.1269668098076925, 0.10069809180846702, 0.10069809180846702, 0.10069809180846678, 0.10069809180846685, 0.10069809180846676, 0.10069809180846669, 0.18636973290336367, 0.1863697329033637, 0.18636973290336384, 0.1863697329033637, 0.18636973290336384, 0.18636973290336384, 0.10476951328179755, 0.10476951328179754, 0.10476951328179747, 0.10476951328179744, 0.10476951328179754, 0.10476951328179743, 0.11807985631483503, 0.1180798563148352, 0.11807985631483502, 0.1180798563148349, 0.11807985631483509, 0.11807985631483503, 0.11792531349738833, 0.11792531349738843, 0.11792531349738837, 0.11792531349738845, 0.1179253134973884, 0.11792531349738833, 0.18437104992516803, 0.18437104992516815, 0.18437104992516803, 0.18437104992516815, 0.18437104992516798, 0.18437104992516798]

Abstraction error: 0.18636973290336384


OVERALL ABSTRACTION ERROR: 0.22883249205446551

Example 6¶

We instantiate a last complex abstraction.

Model definition¶

We redefine the models and the abstraction.

In [39]:

M0 = BN([('A','C'), ('A','D'), ('B','D'), ('B','E'), ('B','F'), ('F','I'), ('D','H'), ('G','E'), ('G','F')])

cpdA = cpd(variable='A',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdB = cpd(variable='B',
          variable_card=4,
          values=generate_values(4,1),
          evidence=None,
          evidence_card=None)

cpdC = cpd(variable='C',
          variable_card=7,
          values=generate_values(7,3),
          evidence=['A'],
          evidence_card=[3])

cpdD = cpd(variable='D',
          variable_card=4,
          values=generate_values(4,12),
          evidence=['A','B'],
          evidence_card=[3,4])

cpdE = cpd(variable='E',
          variable_card=3,
          values=generate_values(3,8),
          evidence=['B','G'],
          evidence_card=[4,2])

cpdF = cpd(variable='F',
          variable_card=3,
          values=generate_values(3,8),
          evidence=['B','G'],
          evidence_card=[4,2])

cpdG = cpd(variable='G',
          variable_card=2,
          values=generate_values(2,1),
          evidence=None,
          evidence_card=None)

cpdH = cpd(variable='H',
          variable_card=6,
          values=generate_values(6,4),
          evidence=['D'],
          evidence_card=[4])

cpdI = cpd(variable='I',
          variable_card=5,
          values=generate_values(5,3),
          evidence=['F'],
          evidence_card=[3])

M0.add_cpds(cpdA,cpdB,cpdC,cpdD,cpdE,cpdF,cpdG,cpdH,cpdI)
M0.check_model()

Out[39]:

True

In [40]:

M1 = BN([('X','Z'), ('Y','W'), ('Y','V')])
M1.add_node('U')

cpdX = cpd(variable='X',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdY = cpd(variable='Y',
          variable_card=3,
          values=generate_values(3,1),
          evidence=None,
          evidence_card=None)

cpdZ = cpd(variable='Z',
          variable_card=24,
          values=generate_values(24,3),
          evidence=['X'],
          evidence_card=[3])

cpdW = cpd(variable='W',
          variable_card=2,
          values=generate_values(2,3),
          evidence=['Y'],
          evidence_card=[3])

cpdV = cpd(variable='V',
          variable_card=8,
          values=generate_values(8,3),
          evidence=['Y'],
          evidence_card=[3])

cpdU = cpd(variable='U',
          variable_card=2,
          values=generate_values(2,1),
          evidence=None,
          evidence_card=None)

M1.add_cpds(cpdX,cpdY,cpdZ,cpdW,cpdV,cpdU)
M1.check_model()

Out[40]:

True

In [41]:

R = ['A','B', 'D', 'E', 'F', 'G', 'H', 'I']

a = {'A': 'X',
     'B': 'Y',
     'D': 'Z',
     'E': 'W',
     'F': 'V',
     'G': 'U',
     'H': 'Z',
     'I': 'V'}
alphas = {'X': generate_values(3,3),
         'Y': generate_values(3,4),
         'Z': generate_values(24,24),
         'W': generate_values(2,3),
         'V': generate_values(8,15),
         'U': generate_values(2,2),
         }

In [42]:

A = Abstraction(M0,M1,R,a,alphas)

Running the computational loop¶

In [43]:

result = A.evaluate_abstraction_error(verbose=True)

- Checking ['X'] -> ['Z']: True
- Checking ['X'] -> ['Y']: False
---- Checking ['A'] -> ['B']: False
- Checking ['X'] -> ['W']: False
---- Checking ['A'] -> ['E']: False
- Checking ['X'] -> ['V']: False
---- Checking ['A'] -> ['F', 'I']: False
- Checking ['X'] -> ['U']: False
---- Checking ['A'] -> ['G']: False
- Checking ['X'] -> ['Z', 'Y']: True
- Checking ['X'] -> ['Z', 'W']: True
- Checking ['X'] -> ['Z', 'V']: True
- Checking ['X'] -> ['Z', 'U']: True
- Checking ['X'] -> ['Y', 'W']: False
---- Checking ['A'] -> ['B', 'E']: False
- Checking ['X'] -> ['Y', 'V']: False
---- Checking ['A'] -> ['B', 'F', 'I']: False
- Checking ['X'] -> ['Y', 'U']: False
---- Checking ['A'] -> ['B', 'G']: False
- Checking ['X'] -> ['W', 'V']: False
---- Checking ['A'] -> ['E', 'F', 'I']: False
- Checking ['X'] -> ['W', 'U']: False
---- Checking ['A'] -> ['E', 'G']: False
- Checking ['X'] -> ['V', 'U']: False
---- Checking ['A'] -> ['F', 'G', 'I']: False
- Checking ['X'] -> ['Z', 'Y', 'W']: True
- Checking ['X'] -> ['Z', 'Y', 'V']: True
- Checking ['X'] -> ['Z', 'Y', 'U']: True
- Checking ['X'] -> ['Z', 'W', 'V']: True
- Checking ['X'] -> ['Z', 'W', 'U']: True
- Checking ['X'] -> ['Z', 'V', 'U']: True
- Checking ['X'] -> ['Y', 'W', 'V']: False
---- Checking ['A'] -> ['B', 'E', 'F', 'I']: False
- Checking ['X'] -> ['Y', 'W', 'U']: False
---- Checking ['A'] -> ['B', 'E', 'G']: False
- Checking ['X'] -> ['Y', 'V', 'U']: False
---- Checking ['A'] -> ['B', 'F', 'G', 'I']: False
- Checking ['X'] -> ['W', 'V', 'U']: False
---- Checking ['A'] -> ['E', 'F', 'G', 'I']: False
- Checking ['X'] -> ['Z', 'Y', 'W', 'V']: True
- Checking ['X'] -> ['Z', 'Y', 'W', 'U']: True
- Checking ['X'] -> ['Z', 'Y', 'V', 'U']: True
- Checking ['X'] -> ['Z', 'W', 'V', 'U']: True
- Checking ['X'] -> ['Y', 'W', 'V', 'U']: False
---- Checking ['A'] -> ['B', 'E', 'F', 'G', 'I']: False
- Checking ['X'] -> ['Z', 'Y', 'W', 'V', 'U']: True
- Checking ['Z'] -> ['X']: False
---- Checking ['D', 'H'] -> ['A']: False
- Checking ['Z'] -> ['Y']: False
---- Checking ['D', 'H'] -> ['B']: False
- Checking ['Z'] -> ['W']: False
---- Checking ['D', 'H'] -> ['E']: False
- Checking ['Z'] -> ['V']: False
---- Checking ['D', 'H'] -> ['F', 'I']: False
- Checking ['Z'] -> ['U']: False
---- Checking ['D', 'H'] -> ['G']: False
- Checking ['Z'] -> ['X', 'Y']: False
---- Checking ['D', 'H'] -> ['A', 'B']: False
- Checking ['Z'] -> ['X', 'W']: False
---- Checking ['D', 'H'] -> ['A', 'E']: False
- Checking ['Z'] -> ['X', 'V']: False
---- Checking ['D', 'H'] -> ['A', 'F', 'I']: False
- Checking ['Z'] -> ['X', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'G']: False
- Checking ['Z'] -> ['Y', 'W']: False
---- Checking ['D', 'H'] -> ['B', 'E']: False
- Checking ['Z'] -> ['Y', 'V']: False
---- Checking ['D', 'H'] -> ['B', 'F', 'I']: False
- Checking ['Z'] -> ['Y', 'U']: False
---- Checking ['D', 'H'] -> ['B', 'G']: False
- Checking ['Z'] -> ['W', 'V']: False
---- Checking ['D', 'H'] -> ['E', 'F', 'I']: False
- Checking ['Z'] -> ['W', 'U']: False
---- Checking ['D', 'H'] -> ['E', 'G']: False
- Checking ['Z'] -> ['V', 'U']: False
---- Checking ['D', 'H'] -> ['F', 'G', 'I']: False
- Checking ['Z'] -> ['X', 'Y', 'W']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'E']: False
- Checking ['Z'] -> ['X', 'Y', 'V']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'F', 'I']: False
- Checking ['Z'] -> ['X', 'Y', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'G']: False
- Checking ['Z'] -> ['X', 'W', 'V']: False
---- Checking ['D', 'H'] -> ['A', 'E', 'F', 'I']: False
- Checking ['Z'] -> ['X', 'W', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'E', 'G']: False
- Checking ['Z'] -> ['X', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'F', 'G', 'I']: False
- Checking ['Z'] -> ['Y', 'W', 'V']: False
---- Checking ['D', 'H'] -> ['B', 'E', 'F', 'I']: False
- Checking ['Z'] -> ['Y', 'W', 'U']: False
---- Checking ['D', 'H'] -> ['B', 'E', 'G']: False
- Checking ['Z'] -> ['Y', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['B', 'F', 'G', 'I']: False
- Checking ['Z'] -> ['W', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['E', 'F', 'G', 'I']: False
- Checking ['Z'] -> ['X', 'Y', 'W', 'V']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'E', 'F', 'I']: False
- Checking ['Z'] -> ['X', 'Y', 'W', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'E', 'G']: False
- Checking ['Z'] -> ['X', 'Y', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'F', 'G', 'I']: False
- Checking ['Z'] -> ['X', 'W', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'E', 'F', 'G', 'I']: False
- Checking ['Z'] -> ['Y', 'W', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['B', 'E', 'F', 'G', 'I']: False
- Checking ['Z'] -> ['X', 'Y', 'W', 'V', 'U']: False
---- Checking ['D', 'H'] -> ['A', 'B', 'E', 'F', 'G', 'I']: False
- Checking ['Y'] -> ['X']: False
---- Checking ['B'] -> ['A']: False
- Checking ['Y'] -> ['Z']: False
---- Checking ['B'] -> ['D', 'H']: True
- Checking ['Y'] -> ['W']: True
- Checking ['Y'] -> ['V']: True
- Checking ['Y'] -> ['U']: False
---- Checking ['B'] -> ['G']: False
- Checking ['Y'] -> ['X', 'Z']: False
---- Checking ['B'] -> ['A', 'D', 'H']: True
- Checking ['Y'] -> ['X', 'W']: True
- Checking ['Y'] -> ['X', 'V']: True
- Checking ['Y'] -> ['X', 'U']: False
---- Checking ['B'] -> ['A', 'G']: False
- Checking ['Y'] -> ['Z', 'W']: True
- Checking ['Y'] -> ['Z', 'V']: True
- Checking ['Y'] -> ['Z', 'U']: False
---- Checking ['B'] -> ['D', 'G', 'H']: True
- Checking ['Y'] -> ['W', 'V']: True
- Checking ['Y'] -> ['W', 'U']: True
- Checking ['Y'] -> ['V', 'U']: True
- Checking ['Y'] -> ['X', 'Z', 'W']: True
- Checking ['Y'] -> ['X', 'Z', 'V']: True
- Checking ['Y'] -> ['X', 'Z', 'U']: False
---- Checking ['B'] -> ['A', 'D', 'G', 'H']: True
- Checking ['Y'] -> ['X', 'W', 'V']: True
- Checking ['Y'] -> ['X', 'W', 'U']: True
- Checking ['Y'] -> ['X', 'V', 'U']: True
- Checking ['Y'] -> ['Z', 'W', 'V']: True
- Checking ['Y'] -> ['Z', 'W', 'U']: True
- Checking ['Y'] -> ['Z', 'V', 'U']: True
- Checking ['Y'] -> ['W', 'V', 'U']: True
- Checking ['Y'] -> ['X', 'Z', 'W', 'V']: True
- Checking ['Y'] -> ['X', 'Z', 'W', 'U']: True
- Checking ['Y'] -> ['X', 'Z', 'V', 'U']: True
- Checking ['Y'] -> ['X', 'W', 'V', 'U']: True
- Checking ['Y'] -> ['Z', 'W', 'V', 'U']: True
- Checking ['Y'] -> ['X', 'Z', 'W', 'V', 'U']: True
- Checking ['W'] -> ['X']: False
---- Checking ['E'] -> ['A']: False
- Checking ['W'] -> ['Z']: False
---- Checking ['E'] -> ['D', 'H']: False
- Checking ['W'] -> ['Y']: False
---- Checking ['E'] -> ['B']: False
- Checking ['W'] -> ['V']: False
---- Checking ['E'] -> ['F', 'I']: False
- Checking ['W'] -> ['U']: False
---- Checking ['E'] -> ['G']: False
- Checking ['W'] -> ['X', 'Z']: False
---- Checking ['E'] -> ['A', 'D', 'H']: False
- Checking ['W'] -> ['X', 'Y']: False
---- Checking ['E'] -> ['A', 'B']: False
- Checking ['W'] -> ['X', 'V']: False
---- Checking ['E'] -> ['A', 'F', 'I']: False
- Checking ['W'] -> ['X', 'U']: False
---- Checking ['E'] -> ['A', 'G']: False
- Checking ['W'] -> ['Z', 'Y']: False
---- Checking ['E'] -> ['B', 'D', 'H']: False
- Checking ['W'] -> ['Z', 'V']: False
---- Checking ['E'] -> ['D', 'F', 'H', 'I']: False
- Checking ['W'] -> ['Z', 'U']: False
---- Checking ['E'] -> ['D', 'G', 'H']: False
- Checking ['W'] -> ['Y', 'V']: False
---- Checking ['E'] -> ['B', 'F', 'I']: False
- Checking ['W'] -> ['Y', 'U']: False
---- Checking ['E'] -> ['B', 'G']: False
- Checking ['W'] -> ['V', 'U']: False
---- Checking ['E'] -> ['F', 'G', 'I']: False
- Checking ['W'] -> ['X', 'Z', 'Y']: False
---- Checking ['E'] -> ['A', 'B', 'D', 'H']: False
- Checking ['W'] -> ['X', 'Z', 'V']: False
---- Checking ['E'] -> ['A', 'D', 'F', 'H', 'I']: False
- Checking ['W'] -> ['X', 'Z', 'U']: False
---- Checking ['E'] -> ['A', 'D', 'G', 'H']: False
- Checking ['W'] -> ['X', 'Y', 'V']: False
---- Checking ['E'] -> ['A', 'B', 'F', 'I']: False
- Checking ['W'] -> ['X', 'Y', 'U']: False
---- Checking ['E'] -> ['A', 'B', 'G']: False
- Checking ['W'] -> ['X', 'V', 'U']: False
---- Checking ['E'] -> ['A', 'F', 'G', 'I']: False
- Checking ['W'] -> ['Z', 'Y', 'V']: False
---- Checking ['E'] -> ['B', 'D', 'F', 'H', 'I']: False
- Checking ['W'] -> ['Z', 'Y', 'U']: False
---- Checking ['E'] -> ['B', 'D', 'G', 'H']: False
- Checking ['W'] -> ['Z', 'V', 'U']: False
---- Checking ['E'] -> ['D', 'F', 'G', 'H', 'I']: False
- Checking ['W'] -> ['Y', 'V', 'U']: False
---- Checking ['E'] -> ['B', 'F', 'G', 'I']: False
- Checking ['W'] -> ['X', 'Z', 'Y', 'V']: False
---- Checking ['E'] -> ['A', 'B', 'D', 'F', 'H', 'I']: False
- Checking ['W'] -> ['X', 'Z', 'Y', 'U']: False
---- Checking ['E'] -> ['A', 'B', 'D', 'G', 'H']: False
- Checking ['W'] -> ['X', 'Z', 'V', 'U']: False
---- Checking ['E'] -> ['A', 'D', 'F', 'G', 'H', 'I']: False
- Checking ['W'] -> ['X', 'Y', 'V', 'U']: False
---- Checking ['E'] -> ['A', 'B', 'F', 'G', 'I']: False
- Checking ['W'] -> ['Z', 'Y', 'V', 'U']: False
---- Checking ['E'] -> ['B', 'D', 'F', 'G', 'H', 'I']: False
- Checking ['W'] -> ['X', 'Z', 'Y', 'V', 'U']: False
---- Checking ['E'] -> ['A', 'B', 'D', 'F', 'G', 'H', 'I']: False
- Checking ['V'] -> ['X']: False
---- Checking ['F', 'I'] -> ['A']: False
- Checking ['V'] -> ['Z']: False
---- Checking ['F', 'I'] -> ['D', 'H']: False
- Checking ['V'] -> ['Y']: False
---- Checking ['F', 'I'] -> ['B']: False
- Checking ['V'] -> ['W']: False
---- Checking ['F', 'I'] -> ['E']: False
- Checking ['V'] -> ['U']: False
---- Checking ['F', 'I'] -> ['G']: False
- Checking ['V'] -> ['X', 'Z']: False
---- Checking ['F', 'I'] -> ['A', 'D', 'H']: False
- Checking ['V'] -> ['X', 'Y']: False
---- Checking ['F', 'I'] -> ['A', 'B']: False
- Checking ['V'] -> ['X', 'W']: False
---- Checking ['F', 'I'] -> ['A', 'E']: False
- Checking ['V'] -> ['X', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'G']: False
- Checking ['V'] -> ['Z', 'Y']: False
---- Checking ['F', 'I'] -> ['B', 'D', 'H']: False
- Checking ['V'] -> ['Z', 'W']: False
---- Checking ['F', 'I'] -> ['D', 'E', 'H']: False
- Checking ['V'] -> ['Z', 'U']: False
---- Checking ['F', 'I'] -> ['D', 'G', 'H']: False
- Checking ['V'] -> ['Y', 'W']: False
---- Checking ['F', 'I'] -> ['B', 'E']: False
- Checking ['V'] -> ['Y', 'U']: False
---- Checking ['F', 'I'] -> ['B', 'G']: False
- Checking ['V'] -> ['W', 'U']: False
---- Checking ['F', 'I'] -> ['E', 'G']: False
- Checking ['V'] -> ['X', 'Z', 'Y']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'D', 'H']: False
- Checking ['V'] -> ['X', 'Z', 'W']: False
---- Checking ['F', 'I'] -> ['A', 'D', 'E', 'H']: False
- Checking ['V'] -> ['X', 'Z', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'D', 'G', 'H']: False
- Checking ['V'] -> ['X', 'Y', 'W']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'E']: False
- Checking ['V'] -> ['X', 'Y', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'G']: False
- Checking ['V'] -> ['X', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'E', 'G']: False
- Checking ['V'] -> ['Z', 'Y', 'W']: False
---- Checking ['F', 'I'] -> ['B', 'D', 'E', 'H']: False
- Checking ['V'] -> ['Z', 'Y', 'U']: False
---- Checking ['F', 'I'] -> ['B', 'D', 'G', 'H']: False
- Checking ['V'] -> ['Z', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['D', 'E', 'G', 'H']: False
- Checking ['V'] -> ['Y', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['B', 'E', 'G']: False
- Checking ['V'] -> ['X', 'Z', 'Y', 'W']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'D', 'E', 'H']: False
- Checking ['V'] -> ['X', 'Z', 'Y', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'D', 'G', 'H']: False
- Checking ['V'] -> ['X', 'Z', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'D', 'E', 'G', 'H']: False
- Checking ['V'] -> ['X', 'Y', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'E', 'G']: False
- Checking ['V'] -> ['Z', 'Y', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['B', 'D', 'E', 'G', 'H']: False
- Checking ['V'] -> ['X', 'Z', 'Y', 'W', 'U']: False
---- Checking ['F', 'I'] -> ['A', 'B', 'D', 'E', 'G', 'H']: False
- Checking ['U'] -> ['X']: False
---- Checking ['G'] -> ['A']: False
- Checking ['U'] -> ['Z']: False
---- Checking ['G'] -> ['D', 'H']: False
- Checking ['U'] -> ['Y']: False
---- Checking ['G'] -> ['B']: False
- Checking ['U'] -> ['W']: False
---- Checking ['G'] -> ['E']: True
- Checking ['U'] -> ['V']: False
---- Checking ['G'] -> ['F', 'I']: True
- Checking ['U'] -> ['X', 'Z']: False
---- Checking ['G'] -> ['A', 'D', 'H']: False
- Checking ['U'] -> ['X', 'Y']: False
---- Checking ['G'] -> ['A', 'B']: False
- Checking ['U'] -> ['X', 'W']: False
---- Checking ['G'] -> ['A', 'E']: True
- Checking ['U'] -> ['X', 'V']: False
---- Checking ['G'] -> ['A', 'F', 'I']: True
- Checking ['U'] -> ['Z', 'Y']: False
---- Checking ['G'] -> ['B', 'D', 'H']: False
- Checking ['U'] -> ['Z', 'W']: False
---- Checking ['G'] -> ['D', 'E', 'H']: True
- Checking ['U'] -> ['Z', 'V']: False
---- Checking ['G'] -> ['D', 'F', 'H', 'I']: True
- Checking ['U'] -> ['Y', 'W']: False
---- Checking ['G'] -> ['B', 'E']: True
- Checking ['U'] -> ['Y', 'V']: False
---- Checking ['G'] -> ['B', 'F', 'I']: True
- Checking ['U'] -> ['W', 'V']: False
---- Checking ['G'] -> ['E', 'F', 'I']: True
- Checking ['U'] -> ['X', 'Z', 'Y']: False
---- Checking ['G'] -> ['A', 'B', 'D', 'H']: False
- Checking ['U'] -> ['X', 'Z', 'W']: False
---- Checking ['G'] -> ['A', 'D', 'E', 'H']: True
- Checking ['U'] -> ['X', 'Z', 'V']: False
---- Checking ['G'] -> ['A', 'D', 'F', 'H', 'I']: True
- Checking ['U'] -> ['X', 'Y', 'W']: False
---- Checking ['G'] -> ['A', 'B', 'E']: True
- Checking ['U'] -> ['X', 'Y', 'V']: False
---- Checking ['G'] -> ['A', 'B', 'F', 'I']: True
- Checking ['U'] -> ['X', 'W', 'V']: False
---- Checking ['G'] -> ['A', 'E', 'F', 'I']: True
- Checking ['U'] -> ['Z', 'Y', 'W']: False
---- Checking ['G'] -> ['B', 'D', 'E', 'H']: True
- Checking ['U'] -> ['Z', 'Y', 'V']: False
---- Checking ['G'] -> ['B', 'D', 'F', 'H', 'I']: True
- Checking ['U'] -> ['Z', 'W', 'V']: False
---- Checking ['G'] -> ['D', 'E', 'F', 'H', 'I']: True
- Checking ['U'] -> ['Y', 'W', 'V']: False
---- Checking ['G'] -> ['B', 'E', 'F', 'I']: True
- Checking ['U'] -> ['X', 'Z', 'Y', 'W']: False
---- Checking ['G'] -> ['A', 'B', 'D', 'E', 'H']: True
- Checking ['U'] -> ['X', 'Z', 'Y', 'V']: False
---- Checking ['G'] -> ['A', 'B', 'D', 'F', 'H', 'I']: True
- Checking ['U'] -> ['X', 'Z', 'W', 'V']: False
---- Checking ['G'] -> ['A', 'D', 'E', 'F', 'H', 'I']: True
- Checking ['U'] -> ['X', 'Y', 'W', 'V']: False
---- Checking ['G'] -> ['A', 'B', 'E', 'F', 'I']: True
- Checking ['U'] -> ['Z', 'Y', 'W', 'V']: False
---- Checking ['G'] -> ['B', 'D', 'E', 'F', 'H', 'I']: True
- Checking ['U'] -> ['X', 'Z', 'Y', 'W', 'V']: False
---- Checking ['G'] -> ['A', 'B', 'D', 'E', 'F', 'H', 'I']: True
- Checking ['X', 'Z'] -> ['Y']: False
---- Checking ['A', 'D', 'H'] -> ['B']: False
- Checking ['X', 'Z'] -> ['W']: False
---- Checking ['A', 'D', 'H'] -> ['E']: False
- Checking ['X', 'Z'] -> ['V']: False
---- Checking ['A', 'D', 'H'] -> ['F', 'I']: False
- Checking ['X', 'Z'] -> ['U']: False
---- Checking ['A', 'D', 'H'] -> ['G']: False
- Checking ['X', 'Z'] -> ['Y', 'W']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'E']: False
- Checking ['X', 'Z'] -> ['Y', 'V']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'F', 'I']: False
- Checking ['X', 'Z'] -> ['Y', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'G']: False
- Checking ['X', 'Z'] -> ['W', 'V']: False
---- Checking ['A', 'D', 'H'] -> ['E', 'F', 'I']: False
- Checking ['X', 'Z'] -> ['W', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['E', 'G']: False
- Checking ['X', 'Z'] -> ['V', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['F', 'G', 'I']: False
- Checking ['X', 'Z'] -> ['Y', 'W', 'V']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'E', 'F', 'I']: False
- Checking ['X', 'Z'] -> ['Y', 'W', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'E', 'G']: False
- Checking ['X', 'Z'] -> ['Y', 'V', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'F', 'G', 'I']: False
- Checking ['X', 'Z'] -> ['W', 'V', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['E', 'F', 'G', 'I']: False
- Checking ['X', 'Z'] -> ['Y', 'W', 'V', 'U']: False
---- Checking ['A', 'D', 'H'] -> ['B', 'E', 'F', 'G', 'I']: False
- Checking ['X', 'Y'] -> ['Z']: True
- Checking ['X', 'Y'] -> ['W']: True
- Checking ['X', 'Y'] -> ['V']: True
- Checking ['X', 'Y'] -> ['U']: False
---- Checking ['A', 'B'] -> ['G']: False
- Checking ['X', 'Y'] -> ['Z', 'W']: True
- Checking ['X', 'Y'] -> ['Z', 'V']: True
- Checking ['X', 'Y'] -> ['Z', 'U']: True
- Checking ['X', 'Y'] -> ['W', 'V']: True
- Checking ['X', 'Y'] -> ['W', 'U']: True
- Checking ['X', 'Y'] -> ['V', 'U']: True
- Checking ['X', 'Y'] -> ['Z', 'W', 'V']: True
- Checking ['X', 'Y'] -> ['Z', 'W', 'U']: True
- Checking ['X', 'Y'] -> ['Z', 'V', 'U']: True
- Checking ['X', 'Y'] -> ['W', 'V', 'U']: True
- Checking ['X', 'Y'] -> ['Z', 'W', 'V', 'U']: True
- Checking ['X', 'W'] -> ['Z']: True
- Checking ['X', 'W'] -> ['Y']: False
---- Checking ['A', 'E'] -> ['B']: False
- Checking ['X', 'W'] -> ['V']: False
---- Checking ['A', 'E'] -> ['F', 'I']: False
- Checking ['X', 'W'] -> ['U']: False
---- Checking ['A', 'E'] -> ['G']: False
- Checking ['X', 'W'] -> ['Z', 'Y']: True
- Checking ['X', 'W'] -> ['Z', 'V']: True
- Checking ['X', 'W'] -> ['Z', 'U']: True
- Checking ['X', 'W'] -> ['Y', 'V']: False
---- Checking ['A', 'E'] -> ['B', 'F', 'I']: False
- Checking ['X', 'W'] -> ['Y', 'U']: False
---- Checking ['A', 'E'] -> ['B', 'G']: False
- Checking ['X', 'W'] -> ['V', 'U']: False
---- Checking ['A', 'E'] -> ['F', 'G', 'I']: False
- Checking ['X', 'W'] -> ['Z', 'Y', 'V']: True
- Checking ['X', 'W'] -> ['Z', 'Y', 'U']: True
- Checking ['X', 'W'] -> ['Z', 'V', 'U']: True
- Checking ['X', 'W'] -> ['Y', 'V', 'U']: False
---- Checking ['A', 'E'] -> ['B', 'F', 'G', 'I']: False
- Checking ['X', 'W'] -> ['Z', 'Y', 'V', 'U']: True
- Checking ['X', 'V'] -> ['Z']: True
- Checking ['X', 'V'] -> ['Y']: False
---- Checking ['A', 'F', 'I'] -> ['B']: False
- Checking ['X', 'V'] -> ['W']: False
---- Checking ['A', 'F', 'I'] -> ['E']: False
- Checking ['X', 'V'] -> ['U']: False
---- Checking ['A', 'F', 'I'] -> ['G']: False
- Checking ['X', 'V'] -> ['Z', 'Y']: True
- Checking ['X', 'V'] -> ['Z', 'W']: True
- Checking ['X', 'V'] -> ['Z', 'U']: True
- Checking ['X', 'V'] -> ['Y', 'W']: False
---- Checking ['A', 'F', 'I'] -> ['B', 'E']: False
- Checking ['X', 'V'] -> ['Y', 'U']: False
---- Checking ['A', 'F', 'I'] -> ['B', 'G']: False
- Checking ['X', 'V'] -> ['W', 'U']: False
---- Checking ['A', 'F', 'I'] -> ['E', 'G']: False
- Checking ['X', 'V'] -> ['Z', 'Y', 'W']: True
- Checking ['X', 'V'] -> ['Z', 'Y', 'U']: True
- Checking ['X', 'V'] -> ['Z', 'W', 'U']: True
- Checking ['X', 'V'] -> ['Y', 'W', 'U']: False
---- Checking ['A', 'F', 'I'] -> ['B', 'E', 'G']: False
- Checking ['X', 'V'] -> ['Z', 'Y', 'W', 'U']: True
- Checking ['X', 'U'] -> ['Z']: True
- Checking ['X', 'U'] -> ['Y']: False
---- Checking ['A', 'G'] -> ['B']: False
- Checking ['X', 'U'] -> ['W']: False
---- Checking ['A', 'G'] -> ['E']: True
- Checking ['X', 'U'] -> ['V']: False
---- Checking ['A', 'G'] -> ['F', 'I']: True
- Checking ['X', 'U'] -> ['Z', 'Y']: True
- Checking ['X', 'U'] -> ['Z', 'W']: True
- Checking ['X', 'U'] -> ['Z', 'V']: True
- Checking ['X', 'U'] -> ['Y', 'W']: False
---- Checking ['A', 'G'] -> ['B', 'E']: True
- Checking ['X', 'U'] -> ['Y', 'V']: False
---- Checking ['A', 'G'] -> ['B', 'F', 'I']: True
- Checking ['X', 'U'] -> ['W', 'V']: False
---- Checking ['A', 'G'] -> ['E', 'F', 'I']: True
- Checking ['X', 'U'] -> ['Z', 'Y', 'W']: True
- Checking ['X', 'U'] -> ['Z', 'Y', 'V']: True
- Checking ['X', 'U'] -> ['Z', 'W', 'V']: True
- Checking ['X', 'U'] -> ['Y', 'W', 'V']: False
---- Checking ['A', 'G'] -> ['B', 'E', 'F', 'I']: True
- Checking ['X', 'U'] -> ['Z', 'Y', 'W', 'V']: True
- Checking ['Z', 'Y'] -> ['X']: False
---- Checking ['B', 'D', 'H'] -> ['A']: False
- Checking ['Z', 'Y'] -> ['W']: True
- Checking ['Z', 'Y'] -> ['V']: True
- Checking ['Z', 'Y'] -> ['U']: False
---- Checking ['B', 'D', 'H'] -> ['G']: False
- Checking ['Z', 'Y'] -> ['X', 'W']: True
- Checking ['Z', 'Y'] -> ['X', 'V']: True
- Checking ['Z', 'Y'] -> ['X', 'U']: False
---- Checking ['B', 'D', 'H'] -> ['A', 'G']: False
- Checking ['Z', 'Y'] -> ['W', 'V']: True
- Checking ['Z', 'Y'] -> ['W', 'U']: True
- Checking ['Z', 'Y'] -> ['V', 'U']: True
- Checking ['Z', 'Y'] -> ['X', 'W', 'V']: True
- Checking ['Z', 'Y'] -> ['X', 'W', 'U']: True
- Checking ['Z', 'Y'] -> ['X', 'V', 'U']: True
- Checking ['Z', 'Y'] -> ['W', 'V', 'U']: True
- Checking ['Z', 'Y'] -> ['X', 'W', 'V', 'U']: True
- Checking ['Z', 'W'] -> ['X']: False
---- Checking ['D', 'E', 'H'] -> ['A']: False
- Checking ['Z', 'W'] -> ['Y']: False
---- Checking ['D', 'E', 'H'] -> ['B']: False
- Checking ['Z', 'W'] -> ['V']: False
---- Checking ['D', 'E', 'H'] -> ['F', 'I']: False
- Checking ['Z', 'W'] -> ['U']: False
---- Checking ['D', 'E', 'H'] -> ['G']: False
- Checking ['Z', 'W'] -> ['X', 'Y']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'B']: False
- Checking ['Z', 'W'] -> ['X', 'V']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'F', 'I']: False
- Checking ['Z', 'W'] -> ['X', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'G']: False
- Checking ['Z', 'W'] -> ['Y', 'V']: False
---- Checking ['D', 'E', 'H'] -> ['B', 'F', 'I']: False
- Checking ['Z', 'W'] -> ['Y', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['B', 'G']: False
- Checking ['Z', 'W'] -> ['V', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['F', 'G', 'I']: False
- Checking ['Z', 'W'] -> ['X', 'Y', 'V']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'B', 'F', 'I']: False
- Checking ['Z', 'W'] -> ['X', 'Y', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'B', 'G']: False
- Checking ['Z', 'W'] -> ['X', 'V', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'F', 'G', 'I']: False
- Checking ['Z', 'W'] -> ['Y', 'V', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['B', 'F', 'G', 'I']: False
- Checking ['Z', 'W'] -> ['X', 'Y', 'V', 'U']: False
---- Checking ['D', 'E', 'H'] -> ['A', 'B', 'F', 'G', 'I']: False
- Checking ['Z', 'V'] -> ['X']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'V'] -> ['Y']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['B']: False
- Checking ['Z', 'V'] -> ['W']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['E']: False
- Checking ['Z', 'V'] -> ['U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['G']: False
- Checking ['Z', 'V'] -> ['X', 'Y']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'B']: False
- Checking ['Z', 'V'] -> ['X', 'W']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'E']: False
- Checking ['Z', 'V'] -> ['X', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'G']: False
- Checking ['Z', 'V'] -> ['Y', 'W']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['B', 'E']: False
- Checking ['Z', 'V'] -> ['Y', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['B', 'G']: False
- Checking ['Z', 'V'] -> ['W', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['E', 'G']: False
- Checking ['Z', 'V'] -> ['X', 'Y', 'W']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'B', 'E']: False
- Checking ['Z', 'V'] -> ['X', 'Y', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'B', 'G']: False
- Checking ['Z', 'V'] -> ['X', 'W', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'E', 'G']: False
- Checking ['Z', 'V'] -> ['Y', 'W', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['B', 'E', 'G']: False
- Checking ['Z', 'V'] -> ['X', 'Y', 'W', 'U']: False
---- Checking ['D', 'F', 'H', 'I'] -> ['A', 'B', 'E', 'G']: False
- Checking ['Z', 'U'] -> ['X']: False
---- Checking ['D', 'G', 'H'] -> ['A']: False
- Checking ['Z', 'U'] -> ['Y']: False
---- Checking ['D', 'G', 'H'] -> ['B']: False
- Checking ['Z', 'U'] -> ['W']: False
---- Checking ['D', 'G', 'H'] -> ['E']: True
- Checking ['Z', 'U'] -> ['V']: False
---- Checking ['D', 'G', 'H'] -> ['F', 'I']: True
- Checking ['Z', 'U'] -> ['X', 'Y']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'B']: False
- Checking ['Z', 'U'] -> ['X', 'W']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'E']: True
- Checking ['Z', 'U'] -> ['X', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'F', 'I']: True
- Checking ['Z', 'U'] -> ['Y', 'W']: False
---- Checking ['D', 'G', 'H'] -> ['B', 'E']: True
- Checking ['Z', 'U'] -> ['Y', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['B', 'F', 'I']: True
- Checking ['Z', 'U'] -> ['W', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['E', 'F', 'I']: True
- Checking ['Z', 'U'] -> ['X', 'Y', 'W']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'B', 'E']: True
- Checking ['Z', 'U'] -> ['X', 'Y', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'B', 'F', 'I']: True
- Checking ['Z', 'U'] -> ['X', 'W', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'E', 'F', 'I']: True
- Checking ['Z', 'U'] -> ['Y', 'W', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['B', 'E', 'F', 'I']: True
- Checking ['Z', 'U'] -> ['X', 'Y', 'W', 'V']: False
---- Checking ['D', 'G', 'H'] -> ['A', 'B', 'E', 'F', 'I']: True
- Checking ['Y', 'W'] -> ['X']: False
---- Checking ['B', 'E'] -> ['A']: False
- Checking ['Y', 'W'] -> ['Z']: False
---- Checking ['B', 'E'] -> ['D', 'H']: True
- Checking ['Y', 'W'] -> ['V']: True
- Checking ['Y', 'W'] -> ['U']: False
---- Checking ['B', 'E'] -> ['G']: False
- Checking ['Y', 'W'] -> ['X', 'Z']: False
---- Checking ['B', 'E'] -> ['A', 'D', 'H']: True
- Checking ['Y', 'W'] -> ['X', 'V']: True
- Checking ['Y', 'W'] -> ['X', 'U']: False
---- Checking ['B', 'E'] -> ['A', 'G']: False
- Checking ['Y', 'W'] -> ['Z', 'V']: True
- Checking ['Y', 'W'] -> ['Z', 'U']: False
---- Checking ['B', 'E'] -> ['D', 'G', 'H']: True
- Checking ['Y', 'W'] -> ['V', 'U']: True
- Checking ['Y', 'W'] -> ['X', 'Z', 'V']: True
- Checking ['Y', 'W'] -> ['X', 'Z', 'U']: False
---- Checking ['B', 'E'] -> ['A', 'D', 'G', 'H']: True
- Checking ['Y', 'W'] -> ['X', 'V', 'U']: True
- Checking ['Y', 'W'] -> ['Z', 'V', 'U']: True
- Checking ['Y', 'W'] -> ['X', 'Z', 'V', 'U']: True
- Checking ['Y', 'V'] -> ['X']: False
---- Checking ['B', 'F', 'I'] -> ['A']: False
- Checking ['Y', 'V'] -> ['Z']: False
---- Checking ['B', 'F', 'I'] -> ['D', 'H']: True
- Checking ['Y', 'V'] -> ['W']: True
- Checking ['Y', 'V'] -> ['U']: False
---- Checking ['B', 'F', 'I'] -> ['G']: False
- Checking ['Y', 'V'] -> ['X', 'Z']: False
---- Checking ['B', 'F', 'I'] -> ['A', 'D', 'H']: True
- Checking ['Y', 'V'] -> ['X', 'W']: True
- Checking ['Y', 'V'] -> ['X', 'U']: False
---- Checking ['B', 'F', 'I'] -> ['A', 'G']: False
- Checking ['Y', 'V'] -> ['Z', 'W']: True
- Checking ['Y', 'V'] -> ['Z', 'U']: False
---- Checking ['B', 'F', 'I'] -> ['D', 'G', 'H']: True
- Checking ['Y', 'V'] -> ['W', 'U']: True
- Checking ['Y', 'V'] -> ['X', 'Z', 'W']: True
- Checking ['Y', 'V'] -> ['X', 'Z', 'U']: False
---- Checking ['B', 'F', 'I'] -> ['A', 'D', 'G', 'H']: True
- Checking ['Y', 'V'] -> ['X', 'W', 'U']: True
- Checking ['Y', 'V'] -> ['Z', 'W', 'U']: True
- Checking ['Y', 'V'] -> ['X', 'Z', 'W', 'U']: True
- Checking ['Y', 'U'] -> ['X']: False
---- Checking ['B', 'G'] -> ['A']: False
- Checking ['Y', 'U'] -> ['Z']: False
---- Checking ['B', 'G'] -> ['D', 'H']: True
- Checking ['Y', 'U'] -> ['W']: True
- Checking ['Y', 'U'] -> ['V']: True
- Checking ['Y', 'U'] -> ['X', 'Z']: False
---- Checking ['B', 'G'] -> ['A', 'D', 'H']: True
- Checking ['Y', 'U'] -> ['X', 'W']: True
- Checking ['Y', 'U'] -> ['X', 'V']: True
- Checking ['Y', 'U'] -> ['Z', 'W']: True
- Checking ['Y', 'U'] -> ['Z', 'V']: True
- Checking ['Y', 'U'] -> ['W', 'V']: True
- Checking ['Y', 'U'] -> ['X', 'Z', 'W']: True
- Checking ['Y', 'U'] -> ['X', 'Z', 'V']: True
- Checking ['Y', 'U'] -> ['X', 'W', 'V']: True
- Checking ['Y', 'U'] -> ['Z', 'W', 'V']: True
- Checking ['Y', 'U'] -> ['X', 'Z', 'W', 'V']: True
- Checking ['W', 'V'] -> ['X']: False
---- Checking ['E', 'F', 'I'] -> ['A']: False
- Checking ['W', 'V'] -> ['Z']: False
---- Checking ['E', 'F', 'I'] -> ['D', 'H']: False
- Checking ['W', 'V'] -> ['Y']: False
---- Checking ['E', 'F', 'I'] -> ['B']: False
- Checking ['W', 'V'] -> ['U']: False
---- Checking ['E', 'F', 'I'] -> ['G']: False
- Checking ['W', 'V'] -> ['X', 'Z']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'D', 'H']: False
- Checking ['W', 'V'] -> ['X', 'Y']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'B']: False
- Checking ['W', 'V'] -> ['X', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'G']: False
- Checking ['W', 'V'] -> ['Z', 'Y']: False
---- Checking ['E', 'F', 'I'] -> ['B', 'D', 'H']: False
- Checking ['W', 'V'] -> ['Z', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['D', 'G', 'H']: False
- Checking ['W', 'V'] -> ['Y', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['B', 'G']: False
- Checking ['W', 'V'] -> ['X', 'Z', 'Y']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'B', 'D', 'H']: False
- Checking ['W', 'V'] -> ['X', 'Z', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'D', 'G', 'H']: False
- Checking ['W', 'V'] -> ['X', 'Y', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'B', 'G']: False
- Checking ['W', 'V'] -> ['Z', 'Y', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['B', 'D', 'G', 'H']: False
- Checking ['W', 'V'] -> ['X', 'Z', 'Y', 'U']: False
---- Checking ['E', 'F', 'I'] -> ['A', 'B', 'D', 'G', 'H']: False
- Checking ['W', 'U'] -> ['X']: False
---- Checking ['E', 'G'] -> ['A']: False
- Checking ['W', 'U'] -> ['Z']: False
---- Checking ['E', 'G'] -> ['D', 'H']: False
- Checking ['W', 'U'] -> ['Y']: False
---- Checking ['E', 'G'] -> ['B']: False
- Checking ['W', 'U'] -> ['V']: False
---- Checking ['E', 'G'] -> ['F', 'I']: True
- Checking ['W', 'U'] -> ['X', 'Z']: False
---- Checking ['E', 'G'] -> ['A', 'D', 'H']: False
- Checking ['W', 'U'] -> ['X', 'Y']: False
---- Checking ['E', 'G'] -> ['A', 'B']: False
- Checking ['W', 'U'] -> ['X', 'V']: False
---- Checking ['E', 'G'] -> ['A', 'F', 'I']: True
- Checking ['W', 'U'] -> ['Z', 'Y']: False
---- Checking ['E', 'G'] -> ['B', 'D', 'H']: False
- Checking ['W', 'U'] -> ['Z', 'V']: False
---- Checking ['E', 'G'] -> ['D', 'F', 'H', 'I']: True
- Checking ['W', 'U'] -> ['Y', 'V']: False
---- Checking ['E', 'G'] -> ['B', 'F', 'I']: True
- Checking ['W', 'U'] -> ['X', 'Z', 'Y']: False
---- Checking ['E', 'G'] -> ['A', 'B', 'D', 'H']: False
- Checking ['W', 'U'] -> ['X', 'Z', 'V']: False
---- Checking ['E', 'G'] -> ['A', 'D', 'F', 'H', 'I']: True
- Checking ['W', 'U'] -> ['X', 'Y', 'V']: False
---- Checking ['E', 'G'] -> ['A', 'B', 'F', 'I']: True
- Checking ['W', 'U'] -> ['Z', 'Y', 'V']: False
---- Checking ['E', 'G'] -> ['B', 'D', 'F', 'H', 'I']: True
- Checking ['W', 'U'] -> ['X', 'Z', 'Y', 'V']: False
---- Checking ['E', 'G'] -> ['A', 'B', 'D', 'F', 'H', 'I']: True
- Checking ['V', 'U'] -> ['X']: False
---- Checking ['F', 'G', 'I'] -> ['A']: False
- Checking ['V', 'U'] -> ['Z']: False
---- Checking ['F', 'G', 'I'] -> ['D', 'H']: False
- Checking ['V', 'U'] -> ['Y']: False
---- Checking ['F', 'G', 'I'] -> ['B']: False
- Checking ['V', 'U'] -> ['W']: False
---- Checking ['F', 'G', 'I'] -> ['E']: True
- Checking ['V', 'U'] -> ['X', 'Z']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'D', 'H']: False
- Checking ['V', 'U'] -> ['X', 'Y']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'B']: False
- Checking ['V', 'U'] -> ['X', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'E']: True
- Checking ['V', 'U'] -> ['Z', 'Y']: False
---- Checking ['F', 'G', 'I'] -> ['B', 'D', 'H']: False
- Checking ['V', 'U'] -> ['Z', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['D', 'E', 'H']: True
- Checking ['V', 'U'] -> ['Y', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['B', 'E']: True
- Checking ['V', 'U'] -> ['X', 'Z', 'Y']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'B', 'D', 'H']: False
- Checking ['V', 'U'] -> ['X', 'Z', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'D', 'E', 'H']: True
- Checking ['V', 'U'] -> ['X', 'Y', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'B', 'E']: True
- Checking ['V', 'U'] -> ['Z', 'Y', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['B', 'D', 'E', 'H']: True
- Checking ['V', 'U'] -> ['X', 'Z', 'Y', 'W']: False
---- Checking ['F', 'G', 'I'] -> ['A', 'B', 'D', 'E', 'H']: True
- Checking ['X', 'Z', 'Y'] -> ['W']: True
- Checking ['X', 'Z', 'Y'] -> ['V']: True
- Checking ['X', 'Z', 'Y'] -> ['U']: False
---- Checking ['A', 'B', 'D', 'H'] -> ['G']: False
- Checking ['X', 'Z', 'Y'] -> ['W', 'V']: True
- Checking ['X', 'Z', 'Y'] -> ['W', 'U']: True
- Checking ['X', 'Z', 'Y'] -> ['V', 'U']: True
- Checking ['X', 'Z', 'Y'] -> ['W', 'V', 'U']: True
- Checking ['X', 'Z', 'W'] -> ['Y']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['B']: False
- Checking ['X', 'Z', 'W'] -> ['V']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['F', 'I']: False
- Checking ['X', 'Z', 'W'] -> ['U']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['G']: False
- Checking ['X', 'Z', 'W'] -> ['Y', 'V']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['B', 'F', 'I']: False
- Checking ['X', 'Z', 'W'] -> ['Y', 'U']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['B', 'G']: False
- Checking ['X', 'Z', 'W'] -> ['V', 'U']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['F', 'G', 'I']: False
- Checking ['X', 'Z', 'W'] -> ['Y', 'V', 'U']: False
---- Checking ['A', 'D', 'E', 'H'] -> ['B', 'F', 'G', 'I']: False
- Checking ['X', 'Z', 'V'] -> ['Y']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['B']: False
- Checking ['X', 'Z', 'V'] -> ['W']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['E']: False
- Checking ['X', 'Z', 'V'] -> ['U']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['G']: False
- Checking ['X', 'Z', 'V'] -> ['Y', 'W']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['B', 'E']: False
- Checking ['X', 'Z', 'V'] -> ['Y', 'U']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['B', 'G']: False
- Checking ['X', 'Z', 'V'] -> ['W', 'U']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['E', 'G']: False
- Checking ['X', 'Z', 'V'] -> ['Y', 'W', 'U']: False
---- Checking ['A', 'D', 'F', 'H', 'I'] -> ['B', 'E', 'G']: False
- Checking ['X', 'Z', 'U'] -> ['Y']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['B']: False
- Checking ['X', 'Z', 'U'] -> ['W']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['E']: True
- Checking ['X', 'Z', 'U'] -> ['V']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['F', 'I']: True
- Checking ['X', 'Z', 'U'] -> ['Y', 'W']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['B', 'E']: True
- Checking ['X', 'Z', 'U'] -> ['Y', 'V']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['B', 'F', 'I']: True
- Checking ['X', 'Z', 'U'] -> ['W', 'V']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['E', 'F', 'I']: True
- Checking ['X', 'Z', 'U'] -> ['Y', 'W', 'V']: False
---- Checking ['A', 'D', 'G', 'H'] -> ['B', 'E', 'F', 'I']: True
- Checking ['X', 'Y', 'W'] -> ['Z']: True
- Checking ['X', 'Y', 'W'] -> ['V']: True
- Checking ['X', 'Y', 'W'] -> ['U']: False
---- Checking ['A', 'B', 'E'] -> ['G']: False
- Checking ['X', 'Y', 'W'] -> ['Z', 'V']: True
- Checking ['X', 'Y', 'W'] -> ['Z', 'U']: True
- Checking ['X', 'Y', 'W'] -> ['V', 'U']: True
- Checking ['X', 'Y', 'W'] -> ['Z', 'V', 'U']: True
- Checking ['X', 'Y', 'V'] -> ['Z']: True
- Checking ['X', 'Y', 'V'] -> ['W']: True
- Checking ['X', 'Y', 'V'] -> ['U']: False
---- Checking ['A', 'B', 'F', 'I'] -> ['G']: False
- Checking ['X', 'Y', 'V'] -> ['Z', 'W']: True
- Checking ['X', 'Y', 'V'] -> ['Z', 'U']: True
- Checking ['X', 'Y', 'V'] -> ['W', 'U']: True
- Checking ['X', 'Y', 'V'] -> ['Z', 'W', 'U']: True
- Checking ['X', 'Y', 'U'] -> ['Z']: True
- Checking ['X', 'Y', 'U'] -> ['W']: True
- Checking ['X', 'Y', 'U'] -> ['V']: True
- Checking ['X', 'Y', 'U'] -> ['Z', 'W']: True
- Checking ['X', 'Y', 'U'] -> ['Z', 'V']: True
- Checking ['X', 'Y', 'U'] -> ['W', 'V']: True
- Checking ['X', 'Y', 'U'] -> ['Z', 'W', 'V']: True
- Checking ['X', 'W', 'V'] -> ['Z']: True
- Checking ['X', 'W', 'V'] -> ['Y']: False
---- Checking ['A', 'E', 'F', 'I'] -> ['B']: False
- Checking ['X', 'W', 'V'] -> ['U']: False
---- Checking ['A', 'E', 'F', 'I'] -> ['G']: False
- Checking ['X', 'W', 'V'] -> ['Z', 'Y']: True
- Checking ['X', 'W', 'V'] -> ['Z', 'U']: True
- Checking ['X', 'W', 'V'] -> ['Y', 'U']: False
---- Checking ['A', 'E', 'F', 'I'] -> ['B', 'G']: False
- Checking ['X', 'W', 'V'] -> ['Z', 'Y', 'U']: True
- Checking ['X', 'W', 'U'] -> ['Z']: True
- Checking ['X', 'W', 'U'] -> ['Y']: False
---- Checking ['A', 'E', 'G'] -> ['B']: False
- Checking ['X', 'W', 'U'] -> ['V']: False
---- Checking ['A', 'E', 'G'] -> ['F', 'I']: True
- Checking ['X', 'W', 'U'] -> ['Z', 'Y']: True
- Checking ['X', 'W', 'U'] -> ['Z', 'V']: True
- Checking ['X', 'W', 'U'] -> ['Y', 'V']: False
---- Checking ['A', 'E', 'G'] -> ['B', 'F', 'I']: True
- Checking ['X', 'W', 'U'] -> ['Z', 'Y', 'V']: True
- Checking ['X', 'V', 'U'] -> ['Z']: True
- Checking ['X', 'V', 'U'] -> ['Y']: False
---- Checking ['A', 'F', 'G', 'I'] -> ['B']: False
- Checking ['X', 'V', 'U'] -> ['W']: False
---- Checking ['A', 'F', 'G', 'I'] -> ['E']: True
- Checking ['X', 'V', 'U'] -> ['Z', 'Y']: True
- Checking ['X', 'V', 'U'] -> ['Z', 'W']: True
- Checking ['X', 'V', 'U'] -> ['Y', 'W']: False
---- Checking ['A', 'F', 'G', 'I'] -> ['B', 'E']: True
- Checking ['X', 'V', 'U'] -> ['Z', 'Y', 'W']: True
- Checking ['Z', 'Y', 'W'] -> ['X']: False
---- Checking ['B', 'D', 'E', 'H'] -> ['A']: False
- Checking ['Z', 'Y', 'W'] -> ['V']: True
- Checking ['Z', 'Y', 'W'] -> ['U']: False
---- Checking ['B', 'D', 'E', 'H'] -> ['G']: False
- Checking ['Z', 'Y', 'W'] -> ['X', 'V']: True
- Checking ['Z', 'Y', 'W'] -> ['X', 'U']: False
---- Checking ['B', 'D', 'E', 'H'] -> ['A', 'G']: False
- Checking ['Z', 'Y', 'W'] -> ['V', 'U']: True
- Checking ['Z', 'Y', 'W'] -> ['X', 'V', 'U']: True
- Checking ['Z', 'Y', 'V'] -> ['X']: False
---- Checking ['B', 'D', 'F', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'Y', 'V'] -> ['W']: True
- Checking ['Z', 'Y', 'V'] -> ['U']: False
---- Checking ['B', 'D', 'F', 'H', 'I'] -> ['G']: False
- Checking ['Z', 'Y', 'V'] -> ['X', 'W']: True
- Checking ['Z', 'Y', 'V'] -> ['X', 'U']: False
---- Checking ['B', 'D', 'F', 'H', 'I'] -> ['A', 'G']: False
- Checking ['Z', 'Y', 'V'] -> ['W', 'U']: True
- Checking ['Z', 'Y', 'V'] -> ['X', 'W', 'U']: True
- Checking ['Z', 'Y', 'U'] -> ['X']: False
---- Checking ['B', 'D', 'G', 'H'] -> ['A']: False
- Checking ['Z', 'Y', 'U'] -> ['W']: True
- Checking ['Z', 'Y', 'U'] -> ['V']: True
- Checking ['Z', 'Y', 'U'] -> ['X', 'W']: True
- Checking ['Z', 'Y', 'U'] -> ['X', 'V']: True
- Checking ['Z', 'Y', 'U'] -> ['W', 'V']: True
- Checking ['Z', 'Y', 'U'] -> ['X', 'W', 'V']: True
- Checking ['Z', 'W', 'V'] -> ['X']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'W', 'V'] -> ['Y']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['B']: False
- Checking ['Z', 'W', 'V'] -> ['U']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['G']: False
- Checking ['Z', 'W', 'V'] -> ['X', 'Y']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['A', 'B']: False
- Checking ['Z', 'W', 'V'] -> ['X', 'U']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['A', 'G']: False
- Checking ['Z', 'W', 'V'] -> ['Y', 'U']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['B', 'G']: False
- Checking ['Z', 'W', 'V'] -> ['X', 'Y', 'U']: False
---- Checking ['D', 'E', 'F', 'H', 'I'] -> ['A', 'B', 'G']: False
- Checking ['Z', 'W', 'U'] -> ['X']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['A']: False
- Checking ['Z', 'W', 'U'] -> ['Y']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['B']: False
- Checking ['Z', 'W', 'U'] -> ['V']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['F', 'I']: True
- Checking ['Z', 'W', 'U'] -> ['X', 'Y']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['A', 'B']: False
- Checking ['Z', 'W', 'U'] -> ['X', 'V']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['A', 'F', 'I']: True
- Checking ['Z', 'W', 'U'] -> ['Y', 'V']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['B', 'F', 'I']: True
- Checking ['Z', 'W', 'U'] -> ['X', 'Y', 'V']: False
---- Checking ['D', 'E', 'G', 'H'] -> ['A', 'B', 'F', 'I']: True
- Checking ['Z', 'V', 'U'] -> ['X']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'V', 'U'] -> ['Y']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['B']: False
- Checking ['Z', 'V', 'U'] -> ['W']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['E']: True
- Checking ['Z', 'V', 'U'] -> ['X', 'Y']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['A', 'B']: False
- Checking ['Z', 'V', 'U'] -> ['X', 'W']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['A', 'E']: True
- Checking ['Z', 'V', 'U'] -> ['Y', 'W']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['B', 'E']: True
- Checking ['Z', 'V', 'U'] -> ['X', 'Y', 'W']: False
---- Checking ['D', 'F', 'G', 'H', 'I'] -> ['A', 'B', 'E']: True
- Checking ['Y', 'W', 'V'] -> ['X']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['A']: False
- Checking ['Y', 'W', 'V'] -> ['Z']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['D', 'H']: True
- Checking ['Y', 'W', 'V'] -> ['U']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['G']: False
- Checking ['Y', 'W', 'V'] -> ['X', 'Z']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['A', 'D', 'H']: True
- Checking ['Y', 'W', 'V'] -> ['X', 'U']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['A', 'G']: False
- Checking ['Y', 'W', 'V'] -> ['Z', 'U']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['D', 'G', 'H']: True
- Checking ['Y', 'W', 'V'] -> ['X', 'Z', 'U']: False
---- Checking ['B', 'E', 'F', 'I'] -> ['A', 'D', 'G', 'H']: True
- Checking ['Y', 'W', 'U'] -> ['X']: False
---- Checking ['B', 'E', 'G'] -> ['A']: False
- Checking ['Y', 'W', 'U'] -> ['Z']: False
---- Checking ['B', 'E', 'G'] -> ['D', 'H']: True
- Checking ['Y', 'W', 'U'] -> ['V']: True
- Checking ['Y', 'W', 'U'] -> ['X', 'Z']: False
---- Checking ['B', 'E', 'G'] -> ['A', 'D', 'H']: True
- Checking ['Y', 'W', 'U'] -> ['X', 'V']: True
- Checking ['Y', 'W', 'U'] -> ['Z', 'V']: True
- Checking ['Y', 'W', 'U'] -> ['X', 'Z', 'V']: True
- Checking ['Y', 'V', 'U'] -> ['X']: False
---- Checking ['B', 'F', 'G', 'I'] -> ['A']: False
- Checking ['Y', 'V', 'U'] -> ['Z']: False
---- Checking ['B', 'F', 'G', 'I'] -> ['D', 'H']: True
- Checking ['Y', 'V', 'U'] -> ['W']: True
- Checking ['Y', 'V', 'U'] -> ['X', 'Z']: False
---- Checking ['B', 'F', 'G', 'I'] -> ['A', 'D', 'H']: True
- Checking ['Y', 'V', 'U'] -> ['X', 'W']: True
- Checking ['Y', 'V', 'U'] -> ['Z', 'W']: True
- Checking ['Y', 'V', 'U'] -> ['X', 'Z', 'W']: True
- Checking ['W', 'V', 'U'] -> ['X']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['A']: False
- Checking ['W', 'V', 'U'] -> ['Z']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['D', 'H']: False
- Checking ['W', 'V', 'U'] -> ['Y']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['B']: False
- Checking ['W', 'V', 'U'] -> ['X', 'Z']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['A', 'D', 'H']: False
- Checking ['W', 'V', 'U'] -> ['X', 'Y']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['A', 'B']: False
- Checking ['W', 'V', 'U'] -> ['Z', 'Y']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['B', 'D', 'H']: False
- Checking ['W', 'V', 'U'] -> ['X', 'Z', 'Y']: False
---- Checking ['E', 'F', 'G', 'I'] -> ['A', 'B', 'D', 'H']: False
- Checking ['X', 'Z', 'Y', 'W'] -> ['V']: True
- Checking ['X', 'Z', 'Y', 'W'] -> ['U']: False
---- Checking ['A', 'B', 'D', 'E', 'H'] -> ['G']: False
- Checking ['X', 'Z', 'Y', 'W'] -> ['V', 'U']: True
- Checking ['X', 'Z', 'Y', 'V'] -> ['W']: True
- Checking ['X', 'Z', 'Y', 'V'] -> ['U']: False
---- Checking ['A', 'B', 'D', 'F', 'H', 'I'] -> ['G']: False
- Checking ['X', 'Z', 'Y', 'V'] -> ['W', 'U']: True
- Checking ['X', 'Z', 'Y', 'U'] -> ['W']: True
- Checking ['X', 'Z', 'Y', 'U'] -> ['V']: True
- Checking ['X', 'Z', 'Y', 'U'] -> ['W', 'V']: True
- Checking ['X', 'Z', 'W', 'V'] -> ['Y']: False
---- Checking ['A', 'D', 'E', 'F', 'H', 'I'] -> ['B']: False
- Checking ['X', 'Z', 'W', 'V'] -> ['U']: False
---- Checking ['A', 'D', 'E', 'F', 'H', 'I'] -> ['G']: False
- Checking ['X', 'Z', 'W', 'V'] -> ['Y', 'U']: False
---- Checking ['A', 'D', 'E', 'F', 'H', 'I'] -> ['B', 'G']: False
- Checking ['X', 'Z', 'W', 'U'] -> ['Y']: False
---- Checking ['A', 'D', 'E', 'G', 'H'] -> ['B']: False
- Checking ['X', 'Z', 'W', 'U'] -> ['V']: False
---- Checking ['A', 'D', 'E', 'G', 'H'] -> ['F', 'I']: True
- Checking ['X', 'Z', 'W', 'U'] -> ['Y', 'V']: False
---- Checking ['A', 'D', 'E', 'G', 'H'] -> ['B', 'F', 'I']: True
- Checking ['X', 'Z', 'V', 'U'] -> ['Y']: False
---- Checking ['A', 'D', 'F', 'G', 'H', 'I'] -> ['B']: False
- Checking ['X', 'Z', 'V', 'U'] -> ['W']: False
---- Checking ['A', 'D', 'F', 'G', 'H', 'I'] -> ['E']: True
- Checking ['X', 'Z', 'V', 'U'] -> ['Y', 'W']: False
---- Checking ['A', 'D', 'F', 'G', 'H', 'I'] -> ['B', 'E']: True
- Checking ['X', 'Y', 'W', 'V'] -> ['Z']: True
- Checking ['X', 'Y', 'W', 'V'] -> ['U']: False
---- Checking ['A', 'B', 'E', 'F', 'I'] -> ['G']: False
- Checking ['X', 'Y', 'W', 'V'] -> ['Z', 'U']: True
- Checking ['X', 'Y', 'W', 'U'] -> ['Z']: True
- Checking ['X', 'Y', 'W', 'U'] -> ['V']: True
- Checking ['X', 'Y', 'W', 'U'] -> ['Z', 'V']: True
- Checking ['X', 'Y', 'V', 'U'] -> ['Z']: True
- Checking ['X', 'Y', 'V', 'U'] -> ['W']: True
- Checking ['X', 'Y', 'V', 'U'] -> ['Z', 'W']: True
- Checking ['X', 'W', 'V', 'U'] -> ['Z']: True
- Checking ['X', 'W', 'V', 'U'] -> ['Y']: False
---- Checking ['A', 'E', 'F', 'G', 'I'] -> ['B']: False
- Checking ['X', 'W', 'V', 'U'] -> ['Z', 'Y']: True
- Checking ['Z', 'Y', 'W', 'V'] -> ['X']: False
---- Checking ['B', 'D', 'E', 'F', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'Y', 'W', 'V'] -> ['U']: False
---- Checking ['B', 'D', 'E', 'F', 'H', 'I'] -> ['G']: False
- Checking ['Z', 'Y', 'W', 'V'] -> ['X', 'U']: False
---- Checking ['B', 'D', 'E', 'F', 'H', 'I'] -> ['A', 'G']: False
- Checking ['Z', 'Y', 'W', 'U'] -> ['X']: False
---- Checking ['B', 'D', 'E', 'G', 'H'] -> ['A']: False
- Checking ['Z', 'Y', 'W', 'U'] -> ['V']: True
- Checking ['Z', 'Y', 'W', 'U'] -> ['X', 'V']: True
- Checking ['Z', 'Y', 'V', 'U'] -> ['X']: False
---- Checking ['B', 'D', 'F', 'G', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'Y', 'V', 'U'] -> ['W']: True
- Checking ['Z', 'Y', 'V', 'U'] -> ['X', 'W']: True
- Checking ['Z', 'W', 'V', 'U'] -> ['X']: False
---- Checking ['D', 'E', 'F', 'G', 'H', 'I'] -> ['A']: False
- Checking ['Z', 'W', 'V', 'U'] -> ['Y']: False
---- Checking ['D', 'E', 'F', 'G', 'H', 'I'] -> ['B']: False
- Checking ['Z', 'W', 'V', 'U'] -> ['X', 'Y']: False
---- Checking ['D', 'E', 'F', 'G', 'H', 'I'] -> ['A', 'B']: False
- Checking ['Y', 'W', 'V', 'U'] -> ['X']: False
---- Checking ['B', 'E', 'F', 'G', 'I'] -> ['A']: False
- Checking ['Y', 'W', 'V', 'U'] -> ['Z']: False
---- Checking ['B', 'E', 'F', 'G', 'I'] -> ['D', 'H']: True
- Checking ['Y', 'W', 'V', 'U'] -> ['X', 'Z']: False
---- Checking ['B', 'E', 'F', 'G', 'I'] -> ['A', 'D', 'H']: True
- Checking ['X', 'Z', 'Y', 'W', 'V'] -> ['U']: False
---- Checking ['A', 'B', 'D', 'E', 'F', 'H', 'I'] -> ['G']: False
- Checking ['X', 'Z', 'Y', 'W', 'U'] -> ['V']: True
- Checking ['X', 'Z', 'Y', 'V', 'U'] -> ['W']: True
- Checking ['X', 'Z', 'W', 'V', 'U'] -> ['Y']: False
---- Checking ['A', 'D', 'E', 'F', 'G', 'H', 'I'] -> ['B']: False
- Checking ['X', 'Y', 'W', 'V', 'U'] -> ['Z']: True
- Checking ['Z', 'Y', 'W', 'V', 'U'] -> ['X']: False
---- Checking ['B', 'D', 'E', 'F', 'G', 'H', 'I'] -> ['A']: False

 305 legitimate pairs of sets out of 3969 possbile pairs of sets

M1: ['X'] -> ['Z']
M0: ['A'] -> ['D', 'H']
M1 mechanism shape: (24, 3)
M0 mechanism shape: (24, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (24, 24)
All JS distances: [0.14671651019565898, 0.12190811938213303, 0.1280838551187487]

Abstraction error: 0.14671651019565898

M1: ['X'] -> ['Z', 'Y']
M0: ['A'] -> ['B', 'D', 'H']
M1 mechanism shape: (72, 3)
M0 mechanism shape: (96, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (72, 96)
All JS distances: [0.17608426608658082, 0.1689907104973962, 0.16789680207707058]

Abstraction error: 0.17608426608658082

M1: ['X'] -> ['Z', 'W']
M0: ['A'] -> ['D', 'E', 'H']
M1 mechanism shape: (48, 3)
M0 mechanism shape: (72, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (48, 72)
All JS distances: [0.2481310745422393, 0.25725674622460787, 0.2634095747826151]

Abstraction error: 0.2634095747826151

M1: ['X'] -> ['Z', 'V']
M0: ['A'] -> ['D', 'F', 'H', 'I']
M1 mechanism shape: (192, 3)
M0 mechanism shape: (360, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (192, 360)
All JS distances: [0.1944475123710473, 0.17935359708817672, 0.1827535077086051]

Abstraction error: 0.1944475123710473

M1: ['X'] -> ['Z', 'U']
M0: ['A'] -> ['D', 'G', 'H']
M1 mechanism shape: (48, 3)
M0 mechanism shape: (48, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (48, 48)
All JS distances: [0.2769616321616567, 0.2661341116393368, 0.2700857185711583]

Abstraction error: 0.2769616321616567

M1: ['X'] -> ['Z', 'Y', 'W']
M0: ['A'] -> ['B', 'D', 'E', 'H']
M1 mechanism shape: (144, 3)
M0 mechanism shape: (288, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (144, 288)
All JS distances: [0.3427808206762295, 0.35419773536584853, 0.35522524503406877]

Abstraction error: 0.35522524503406877

M1: ['X'] -> ['Z', 'Y', 'V']
M0: ['A'] -> ['B', 'D', 'F', 'H', 'I']
M1 mechanism shape: (576, 3)
M0 mechanism shape: (1440, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (576, 1440)
All JS distances: [0.28110674920375944, 0.2720271820224432, 0.2731384357167785]

Abstraction error: 0.28110674920375944

M1: ['X'] -> ['Z', 'Y', 'U']
M0: ['A'] -> ['B', 'D', 'G', 'H']
M1 mechanism shape: (144, 3)
M0 mechanism shape: (192, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (144, 192)
All JS distances: [0.298556147369432, 0.2902566519104544, 0.2947506608063759]

Abstraction error: 0.298556147369432

M1: ['X'] -> ['Z', 'W', 'V']
M0: ['A'] -> ['D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (384, 3)
M0 mechanism shape: (1080, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (384, 1080)
All JS distances: [0.3182422189515313, 0.3102018627433937, 0.3128511714646684]

Abstraction error: 0.3182422189515313

M1: ['X'] -> ['Z', 'W', 'U']
M0: ['A'] -> ['D', 'E', 'G', 'H']
M1 mechanism shape: (96, 3)
M0 mechanism shape: (144, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (96, 144)
All JS distances: [0.34968192148698235, 0.34135716378151104, 0.34625050188033957]

Abstraction error: 0.34968192148698235

M1: ['X'] -> ['Z', 'V', 'U']
M0: ['A'] -> ['D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (384, 3)
M0 mechanism shape: (720, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (384, 720)
All JS distances: [0.29864578015620025, 0.29179360402924703, 0.2929775776422845]

Abstraction error: 0.29864578015620025

M1: ['X'] -> ['Z', 'Y', 'W', 'V']
M0: ['A'] -> ['B', 'D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (1152, 3)
M0 mechanism shape: (4320, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (1152, 4320)
All JS distances: [0.4163482355016841, 0.41182307768684834, 0.4125983745907082]

Abstraction error: 0.4163482355016841

M1: ['X'] -> ['Z', 'Y', 'W', 'U']
M0: ['A'] -> ['B', 'D', 'E', 'G', 'H']
M1 mechanism shape: (288, 3)
M0 mechanism shape: (576, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (288, 576)
All JS distances: [0.41382556346877014, 0.41000991689621746, 0.41300266579057804]

Abstraction error: 0.41382556346877014

M1: ['X'] -> ['Z', 'Y', 'V', 'U']
M0: ['A'] -> ['B', 'D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (1152, 3)
M0 mechanism shape: (2880, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (1152, 2880)
All JS distances: [0.35028191225451477, 0.34679695557911117, 0.347648699875281]

Abstraction error: 0.35028191225451477

M1: ['X'] -> ['Z', 'W', 'V', 'U']
M0: ['A'] -> ['D', 'E', 'F', 'G', 'H', 'I']
M1 mechanism shape: (768, 3)
M0 mechanism shape: (2160, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (768, 2160)
All JS distances: [0.3959288005789213, 0.3911860880350616, 0.3924179184638522]

Abstraction error: 0.3959288005789213

M1: ['X'] -> ['Z', 'Y', 'W', 'V', 'U']
M0: ['A'] -> ['B', 'D', 'E', 'F', 'G', 'H', 'I']
M1 mechanism shape: (2304, 3)
M0 mechanism shape: (8640, 3)
Alpha_s shape: (3, 3)
Alpha_t shape: (2304, 8640)
All JS distances: [0.469134837136592, 0.4665992960764694, 0.46671166935350306]

Abstraction error: 0.469134837136592

M1: ['Y'] -> ['Z']
M0: ['B'] -> ['D', 'H']
M1 mechanism shape: (24, 3)
M0 mechanism shape: (24, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (24, 24)
All JS distances: [0.14583320915276315, 0.1418612126809944, 0.1399368402225962, 0.14625835179015392]

Abstraction error: 0.14625835179015392

M1: ['Y'] -> ['W']
M0: ['B'] -> ['E']
M1 mechanism shape: (2, 3)
M0 mechanism shape: (3, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (2, 3)
All JS distances: [0.3243736203962745, 0.1735419811534571, 0.3394418600185987, 0.21218306043526702]

Abstraction error: 0.3394418600185987

M1: ['Y'] -> ['V']
M0: ['B'] -> ['F', 'I']
M1 mechanism shape: (8, 3)
M0 mechanism shape: (15, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (8, 15)
All JS distances: [0.12059895477285079, 0.16748634037318327, 0.12175151889575896, 0.09189169585012583]

Abstraction error: 0.16748634037318327

M1: ['Y'] -> ['X', 'Z']
M0: ['B'] -> ['A', 'D', 'H']
M1 mechanism shape: (72, 3)
M0 mechanism shape: (72, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (72, 72)
All JS distances: [0.32775728551966704, 0.3254143942633815, 0.3242808903866604, 0.3268079445212499]

Abstraction error: 0.32775728551966704

M1: ['Y'] -> ['X', 'W']
M0: ['B'] -> ['A', 'E']
M1 mechanism shape: (6, 3)
M0 mechanism shape: (9, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (6, 9)
All JS distances: [0.4088053365640753, 0.3205291540735976, 0.41918557822044117, 0.339807615261772]

Abstraction error: 0.41918557822044117

M1: ['Y'] -> ['X', 'V']
M0: ['B'] -> ['A', 'F', 'I']
M1 mechanism shape: (24, 3)
M0 mechanism shape: (45, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (24, 45)
All JS distances: [0.2991638243761056, 0.318072573754108, 0.29956600113704535, 0.29032357801221254]

Abstraction error: 0.318072573754108

M1: ['Y'] -> ['Z', 'W']
M0: ['B'] -> ['D', 'E', 'H']
M1 mechanism shape: (48, 3)
M0 mechanism shape: (72, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (48, 72)
All JS distances: [0.3136429437058418, 0.17361831087761584, 0.31706875141028595, 0.20016164165016945]

Abstraction error: 0.31706875141028595

M1: ['Y'] -> ['Z', 'V']
M0: ['B'] -> ['D', 'F', 'H', 'I']
M1 mechanism shape: (192, 3)
M0 mechanism shape: (360, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (192, 360)
All JS distances: [0.1919685284578916, 0.21697256295316963, 0.1892891707709742, 0.1697555341429652]

Abstraction error: 0.21697256295316963

M1: ['Y'] -> ['Z', 'U']
M0: ['B'] -> ['D', 'G', 'H']
M1 mechanism shape: (48, 3)
M0 mechanism shape: (48, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (48, 48)
All JS distances: [0.2782700468611738, 0.2753741929968432, 0.2742468058468797, 0.27534766804696686]

Abstraction error: 0.2782700468611738

M1: ['Y'] -> ['W', 'V']
M0: ['B'] -> ['E', 'F', 'I']
M1 mechanism shape: (16, 3)
M0 mechanism shape: (45, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (16, 45)
All JS distances: [0.34912516804539734, 0.23789878129489664, 0.3641392077843359, 0.24240421566679024]

Abstraction error: 0.3641392077843359

M1: ['Y'] -> ['W', 'U']
M0: ['B'] -> ['E', 'G']
M1 mechanism shape: (4, 3)
M0 mechanism shape: (6, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (4, 6)
All JS distances: [0.39438320471460564, 0.2972419273221465, 0.40425178887785795, 0.3187800630994384]

Abstraction error: 0.40425178887785795

M1: ['Y'] -> ['V', 'U']
M0: ['B'] -> ['F', 'G', 'I']
M1 mechanism shape: (16, 3)
M0 mechanism shape: (30, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (16, 30)
All JS distances: [0.258593655183137, 0.287956919365726, 0.26220805003951686, 0.2542800363833757]

Abstraction error: 0.287956919365726

M1: ['Y'] -> ['X', 'Z', 'W']
M0: ['B'] -> ['A', 'D', 'E', 'H']
M1 mechanism shape: (144, 3)
M0 mechanism shape: (216, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (144, 216)
All JS distances: [0.4145702447669112, 0.3381252910853169, 0.4166161587940429, 0.3506858367124328]

Abstraction error: 0.4166161587940429

M1: ['Y'] -> ['X', 'Z', 'V']
M0: ['B'] -> ['A', 'D', 'F', 'H', 'I']
M1 mechanism shape: (576, 3)
M0 mechanism shape: (1080, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (576, 1080)
All JS distances: [0.3469962948268254, 0.3589501830906501, 0.34498338295932235, 0.33645274166455186]

Abstraction error: 0.3589501830906501

M1: ['Y'] -> ['X', 'Z', 'U']
M0: ['B'] -> ['A', 'D', 'G', 'H']
M1 mechanism shape: (144, 3)
M0 mechanism shape: (144, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (144, 144)
All JS distances: [0.3927709412636983, 0.3907152211904342, 0.3898061950345306, 0.3905816898721179]

Abstraction error: 0.3927709412636983

M1: ['Y'] -> ['X', 'W', 'V']
M0: ['B'] -> ['A', 'E', 'F', 'I']
M1 mechanism shape: (48, 3)
M0 mechanism shape: (135, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (48, 135)
All JS distances: [0.42636337837605315, 0.3544161636639346, 0.43710601002863136, 0.3568584747625011]

Abstraction error: 0.43710601002863136

M1: ['Y'] -> ['X', 'W', 'U']
M0: ['B'] -> ['A', 'E', 'G']
M1 mechanism shape: (12, 3)
M0 mechanism shape: (18, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (12, 18)
All JS distances: [0.459607215109885, 0.39097437651142114, 0.4670078090389173, 0.4054226747920224]

Abstraction error: 0.4670078090389173

M1: ['Y'] -> ['X', 'V', 'U']
M0: ['B'] -> ['A', 'F', 'G', 'I']
M1 mechanism shape: (48, 3)
M0 mechanism shape: (90, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (48, 90)
All JS distances: [0.36648187670696886, 0.3850431423662941, 0.368739186470645, 0.36384918986539994]

Abstraction error: 0.3850431423662941

M1: ['Y'] -> ['Z', 'W', 'V']
M0: ['B'] -> ['D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (384, 3)
M0 mechanism shape: (1080, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (384, 1080)
All JS distances: [0.34083486006772207, 0.2384598477865502, 0.3478641308744728, 0.2669749911396077]

Abstraction error: 0.3478641308744728

M1: ['Y'] -> ['Z', 'W', 'U']
M0: ['B'] -> ['D', 'E', 'G', 'H']
M1 mechanism shape: (96, 3)
M0 mechanism shape: (144, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (96, 144)
All JS distances: [0.38333004375026625, 0.29398511372149194, 0.37996832411448994, 0.31466128299842316]

Abstraction error: 0.38333004375026625

M1: ['Y'] -> ['Z', 'V', 'U']
M0: ['B'] -> ['D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (384, 3)
M0 mechanism shape: (720, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (384, 720)
All JS distances: [0.29567947707217684, 0.3152981514416879, 0.2962552570056811, 0.28740269791748396]

Abstraction error: 0.3152981514416879

M1: ['Y'] -> ['W', 'V', 'U']
M0: ['B'] -> ['E', 'F', 'G', 'I']
M1 mechanism shape: (32, 3)
M0 mechanism shape: (90, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (32, 90)
All JS distances: [0.40542677119257703, 0.32937410423810237, 0.41871107268883256, 0.33031006600997415]

Abstraction error: 0.41871107268883256

M1: ['Y'] -> ['X', 'Z', 'W', 'V']
M0: ['B'] -> ['A', 'D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (1152, 3)
M0 mechanism shape: (3240, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (1152, 3240)
All JS distances: [0.43259328144322, 0.37023857049128744, 0.436655534784244, 0.38591371156834237]

Abstraction error: 0.436655534784244

M1: ['Y'] -> ['X', 'Z', 'W', 'U']
M0: ['B'] -> ['A', 'D', 'E', 'G', 'H']
M1 mechanism shape: (288, 3)
M0 mechanism shape: (432, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (288, 432)
All JS distances: [0.4617143875394613, 0.4020904921288046, 0.4591208669291066, 0.4151818879003503]

Abstraction error: 0.4617143875394613

M1: ['Y'] -> ['X', 'Z', 'V', 'U']
M0: ['B'] -> ['A', 'D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (1152, 3)
M0 mechanism shape: (2160, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (1152, 2160)
All JS distances: [0.40373896420788313, 0.4161271411198629, 0.4037902853702875, 0.39775628667449914]

Abstraction error: 0.4161271411198629

M1: ['Y'] -> ['X', 'W', 'V', 'U']
M0: ['B'] -> ['A', 'E', 'F', 'G', 'I']
M1 mechanism shape: (96, 3)
M0 mechanism shape: (270, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (96, 270)
All JS distances: [0.4680222241240685, 0.4128462190587115, 0.4781360723756702, 0.4134293189802729]

Abstraction error: 0.4781360723756702

M1: ['Y'] -> ['Z', 'W', 'V', 'U']
M0: ['B'] -> ['D', 'E', 'F', 'G', 'H', 'I']
M1 mechanism shape: (768, 3)
M0 mechanism shape: (2160, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (768, 2160)
All JS distances: [0.4008073857031001, 0.3270188329370278, 0.45149062810817364, 0.33705791965972]

Abstraction error: 0.45149062810817364

M1: ['Y'] -> ['X', 'Z', 'W', 'V', 'U']
M0: ['B'] -> ['A', 'D', 'E', 'F', 'G', 'H', 'I']
M1 mechanism shape: (2304, 3)
M0 mechanism shape: (6480, 4)
Alpha_s shape: (3, 4)
Alpha_t shape: (2304, 6480)
All JS distances: [0.4745705327622316, 0.4235448437567008, 0.5115628136399912, 0.429896561782647]

Abstraction error: 0.5115628136399912

M1: ['U'] -> ['W']
M0: ['G'] -> ['E']
M1 mechanism shape: (2, 2)
M0 mechanism shape: (3, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (2, 3)
All JS distances: [0.28365370223347247, 0.28282402813485186]

Abstraction error: 0.28365370223347247

M1: ['U'] -> ['V']
M0: ['G'] -> ['F', 'I']
M1 mechanism shape: (8, 2)
M0 mechanism shape: (15, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (8, 15)
All JS distances: [0.12491355211742401, 0.12508085160141344]

Abstraction error: 0.12508085160141344

M1: ['U'] -> ['X', 'W']
M0: ['G'] -> ['A', 'E']
M1 mechanism shape: (6, 2)
M0 mechanism shape: (9, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (6, 9)
All JS distances: [0.38185667423989283, 0.38132934312009487]

Abstraction error: 0.38185667423989283

M1: ['U'] -> ['X', 'V']
M0: ['G'] -> ['A', 'F', 'I']
M1 mechanism shape: (24, 2)
M0 mechanism shape: (45, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (24, 45)
All JS distances: [0.3006838209893557, 0.30074920378752945]

Abstraction error: 0.30074920378752945

M1: ['U'] -> ['Z', 'W']
M0: ['G'] -> ['D', 'E', 'H']
M1 mechanism shape: (48, 2)
M0 mechanism shape: (72, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (48, 72)
All JS distances: [0.260256984969388, 0.2611099843496716]

Abstraction error: 0.2611099843496716

M1: ['U'] -> ['Z', 'V']
M0: ['G'] -> ['D', 'F', 'H', 'I']
M1 mechanism shape: (192, 2)
M0 mechanism shape: (360, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (192, 360)
All JS distances: [0.19043043815867075, 0.19432657207417994]

Abstraction error: 0.19432657207417994

M1: ['U'] -> ['Y', 'W']
M0: ['G'] -> ['B', 'E']
M1 mechanism shape: (6, 2)
M0 mechanism shape: (12, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (6, 12)
All JS distances: [0.3822774086698545, 0.37921353595613094]

Abstraction error: 0.3822774086698545

M1: ['U'] -> ['Y', 'V']
M0: ['G'] -> ['B', 'F', 'I']
M1 mechanism shape: (24, 2)
M0 mechanism shape: (60, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (24, 60)
All JS distances: [0.254750211201766, 0.2560743634951334]

Abstraction error: 0.2560743634951334

M1: ['U'] -> ['W', 'V']
M0: ['G'] -> ['E', 'F', 'I']
M1 mechanism shape: (16, 2)
M0 mechanism shape: (45, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (16, 45)
All JS distances: [0.31565860912713517, 0.3145615145859717]

Abstraction error: 0.31565860912713517

M1: ['U'] -> ['X', 'Z', 'W']
M0: ['G'] -> ['A', 'D', 'E', 'H']
M1 mechanism shape: (144, 2)
M0 mechanism shape: (216, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (144, 216)
All JS distances: [0.38222814199997895, 0.3825982298828456]

Abstraction error: 0.3825982298828456

M1: ['U'] -> ['X', 'Z', 'V']
M0: ['G'] -> ['A', 'D', 'F', 'H', 'I']
M1 mechanism shape: (576, 2)
M0 mechanism shape: (1080, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (576, 1080)
All JS distances: [0.34559852340491337, 0.34728564678116325]

Abstraction error: 0.34728564678116325

M1: ['U'] -> ['X', 'Y', 'W']
M0: ['G'] -> ['A', 'B', 'E']
M1 mechanism shape: (18, 2)
M0 mechanism shape: (36, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (18, 36)
All JS distances: [0.45148928028206436, 0.44924413091861465]

Abstraction error: 0.45148928028206436

M1: ['U'] -> ['X', 'Y', 'V']
M0: ['G'] -> ['A', 'B', 'F', 'I']
M1 mechanism shape: (72, 2)
M0 mechanism shape: (180, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (72, 180)
All JS distances: [0.3643987151500383, 0.3651824442016619]

Abstraction error: 0.3651824442016619

M1: ['U'] -> ['X', 'W', 'V']
M0: ['G'] -> ['A', 'E', 'F', 'I']
M1 mechanism shape: (48, 2)
M0 mechanism shape: (135, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (48, 135)
All JS distances: [0.40328304196904785, 0.40254549259952893]

Abstraction error: 0.40328304196904785

M1: ['U'] -> ['Z', 'Y', 'W']
M0: ['G'] -> ['B', 'D', 'E', 'H']
M1 mechanism shape: (144, 2)
M0 mechanism shape: (288, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (144, 288)
All JS distances: [0.352404290336872, 0.35244234740334995]

Abstraction error: 0.35244234740334995

M1: ['U'] -> ['Z', 'Y', 'V']
M0: ['G'] -> ['B', 'D', 'F', 'H', 'I']
M1 mechanism shape: (576, 2)
M0 mechanism shape: (1440, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (576, 1440)
All JS distances: [0.2759660114022123, 0.27978874364078304]

Abstraction error: 0.27978874364078304

M1: ['U'] -> ['Z', 'W', 'V']
M0: ['G'] -> ['D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (384, 2)
M0 mechanism shape: (1080, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (384, 1080)
All JS distances: [0.3174489646385951, 0.31580810413130855]

Abstraction error: 0.3174489646385951

M1: ['U'] -> ['Y', 'W', 'V']
M0: ['G'] -> ['B', 'E', 'F', 'I']
M1 mechanism shape: (48, 2)
M0 mechanism shape: (180, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (48, 180)
All JS distances: [0.42271211321187113, 0.4203751677620154]

Abstraction error: 0.42271211321187113

M1: ['U'] -> ['X', 'Z', 'Y', 'W']
M0: ['G'] -> ['A', 'B', 'D', 'E', 'H']
M1 mechanism shape: (432, 2)
M0 mechanism shape: (864, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (432, 864)
All JS distances: [0.44183930491280465, 0.441866926184124]

Abstraction error: 0.441866926184124

M1: ['U'] -> ['X', 'Z', 'Y', 'V']
M0: ['G'] -> ['A', 'B', 'D', 'F', 'H', 'I']
M1 mechanism shape: (1728, 2)
M0 mechanism shape: (4320, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (1728, 4320)
All JS distances: [0.3916760158239113, 0.39381052591623633]

Abstraction error: 0.39381052591623633

M1: ['U'] -> ['X', 'Z', 'W', 'V']
M0: ['G'] -> ['A', 'D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (1152, 2)
M0 mechanism shape: (3240, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (1152, 3240)
All JS distances: [0.41687719829017855, 0.4157440975592888]

Abstraction error: 0.41687719829017855

M1: ['U'] -> ['X', 'Y', 'W', 'V']
M0: ['G'] -> ['A', 'B', 'E', 'F', 'I']
M1 mechanism shape: (144, 2)
M0 mechanism shape: (540, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (144, 540)
All JS distances: [0.4819880038368549, 0.4802087212167013]

Abstraction error: 0.4819880038368549

M1: ['U'] -> ['Z', 'Y', 'W', 'V']
M0: ['G'] -> ['B', 'D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (1152, 2)
M0 mechanism shape: (4320, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (1152, 4320)
All JS distances: [0.4172639512227314, 0.40962384323107615]

Abstraction error: 0.4172639512227314

M1: ['U'] -> ['X', 'Z', 'Y', 'W', 'V']
M0: ['G'] -> ['A', 'B', 'D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (3456, 2)
M0 mechanism shape: (12960, 2)
Alpha_s shape: (2, 2)
Alpha_t shape: (3456, 12960)
All JS distances: [0.48710061910726293, 0.48155128640502154]

Abstraction error: 0.48710061910726293

M1: ['X', 'Y'] -> ['Z']
M0: ['A', 'B'] -> ['D', 'H']
M1 mechanism shape: (24, 9)
M0 mechanism shape: (24, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (24, 24)
All JS distances: [0.14750258547137535, 0.14477921931232055, 0.14686181058189657, 0.15010120315965594, 0.14322994822209403, 0.12280811148432946, 0.11942718073653451, 0.12956066510934938, 0.12060711691901498, 0.13979964359918703, 0.1269272990419471, 0.1298119712581675]

Abstraction error: 0.15010120315965594

M1: ['X', 'Y'] -> ['W']
M0: ['A', 'B'] -> ['E']
M1 mechanism shape: (2, 9)
M0 mechanism shape: (3, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (2, 3)
All JS distances: [0.32437362039627454, 0.17354198115345706, 0.3394418600185988, 0.21218306043526702, 0.32437362039627426, 0.17354198115345687, 0.33944186001859855, 0.21218306043526705, 0.3243736203962744, 0.1735419811534571, 0.3394418600185988, 0.21218306043526702]

Abstraction error: 0.3394418600185988

M1: ['X', 'Y'] -> ['V']
M0: ['A', 'B'] -> ['F', 'I']
M1 mechanism shape: (8, 9)
M0 mechanism shape: (15, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (8, 15)
All JS distances: [0.12059895477285078, 0.1674863403731833, 0.12175151889575926, 0.09189169585012574, 0.12059895477285067, 0.16748634037318336, 0.12175151889575915, 0.09189169585012594, 0.12059895477285074, 0.16748634037318347, 0.12175151889575897, 0.0918916958501261]

Abstraction error: 0.16748634037318347

M1: ['X', 'Y'] -> ['Z', 'W']
M0: ['A', 'B'] -> ['D', 'E', 'H']
M1 mechanism shape: (48, 9)
M0 mechanism shape: (72, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (48, 72)
All JS distances: [0.31524025575465625, 0.18634194563767592, 0.3068898074906512, 0.18584477657549822, 0.3132215906738382, 0.15484809565944999, 0.3168915795156973, 0.19559968618896081, 0.29791839666715264, 0.1742214982251845, 0.3144643331109569, 0.20593677218425346]

Abstraction error: 0.3168915795156973

M1: ['X', 'Y'] -> ['Z', 'V']
M0: ['A', 'B'] -> ['D', 'F', 'H', 'I']
M1 mechanism shape: (192, 9)
M0 mechanism shape: (360, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (192, 360)
All JS distances: [0.19127630844616872, 0.21842778713325303, 0.19388609720299727, 0.1708797666794156, 0.18857546277248374, 0.2078468475138132, 0.17641278895149123, 0.1579332787626251, 0.177806384307109, 0.21388967790232216, 0.1811877424824734, 0.1583250780651464]

Abstraction error: 0.21842778713325303

M1: ['X', 'Y'] -> ['Z', 'U']
M0: ['A', 'B'] -> ['D', 'G', 'H']
M1 mechanism shape: (48, 9)
M0 mechanism shape: (48, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (48, 48)
All JS distances: [0.28298482901192257, 0.2800907217618497, 0.2787437643486997, 0.2758966179519917, 0.27656746522310505, 0.2656310464799127, 0.26617428345832916, 0.26843710086540823, 0.26483723363459705, 0.2753472579922986, 0.2683612765301676, 0.2720343370480691]

Abstraction error: 0.28298482901192257

M1: ['X', 'Y'] -> ['W', 'V']
M0: ['A', 'B'] -> ['E', 'F', 'I']
M1 mechanism shape: (16, 9)
M0 mechanism shape: (45, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (16, 45)
All JS distances: [0.3491251680453974, 0.23789878129489667, 0.3641392077843359, 0.24240421566679018, 0.34912516804539734, 0.23789878129489664, 0.3641392077843359, 0.24240421566679018, 0.34912516804539734, 0.23789878129489664, 0.36413920778433584, 0.24240421566679024]

Abstraction error: 0.3641392077843359

M1: ['X', 'Y'] -> ['W', 'U']
M0: ['A', 'B'] -> ['E', 'G']
M1 mechanism shape: (4, 9)
M0 mechanism shape: (6, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (4, 6)
All JS distances: [0.3943832047146057, 0.2972419273221465, 0.404251788877858, 0.31878006309943835, 0.39438320471460564, 0.2972419273221465, 0.404251788877858, 0.31878006309943846, 0.3943832047146056, 0.2972419273221465, 0.40425178887785806, 0.3187800630994384]

Abstraction error: 0.40425178887785806

M1: ['X', 'Y'] -> ['V', 'U']
M0: ['A', 'B'] -> ['F', 'G', 'I']
M1 mechanism shape: (16, 9)
M0 mechanism shape: (30, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (16, 30)
All JS distances: [0.258593655183137, 0.28795691936572604, 0.26220805003951686, 0.2542800363833757, 0.25859365518313704, 0.28795691936572604, 0.26220805003951686, 0.25428003638337576, 0.25859365518313693, 0.287956919365726, 0.26220805003951686, 0.25428003638337576]

Abstraction error: 0.28795691936572604

M1: ['X', 'Y'] -> ['Z', 'W', 'V']
M0: ['A', 'B'] -> ['D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (384, 9)
M0 mechanism shape: (1080, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (384, 1080)
All JS distances: [0.34022697261416096, 0.24241224903656333, 0.3497961649925819, 0.2675106158112102, 0.3395257480812267, 0.22930779072855179, 0.3413544789160718, 0.2607416132595406, 0.33414063854706944, 0.23972072538614778, 0.3446542645636457, 0.2615743256681776]

Abstraction error: 0.3497961649925819

M1: ['X', 'Y'] -> ['Z', 'W', 'U']
M0: ['A', 'B'] -> ['D', 'E', 'G', 'H']
M1 mechanism shape: (96, 9)
M0 mechanism shape: (144, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (96, 144)
All JS distances: [0.38928149096433073, 0.3018455521060331, 0.3844350318863644, 0.3141461569097123, 0.3825734716938089, 0.2834151290160743, 0.37429119504763586, 0.30899404315320994, 0.37311535674244317, 0.2973425794468723, 0.3763292860989567, 0.3144067715994976]

Abstraction error: 0.38928149096433073

M1: ['X', 'Y'] -> ['Z', 'V', 'U']
M0: ['A', 'B'] -> ['D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (384, 9)
M0 mechanism shape: (720, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (384, 720)
All JS distances: [0.29497669457620096, 0.316324564958404, 0.29961428299158566, 0.2877808054067474, 0.2942458014279417, 0.31179855929673794, 0.2907285234983983, 0.28087635232985686, 0.2923188145601126, 0.31315047879600083, 0.29384754091440074, 0.2799297081009121]

Abstraction error: 0.316324564958404

M1: ['X', 'Y'] -> ['W', 'V', 'U']
M0: ['A', 'B'] -> ['E', 'F', 'G', 'I']
M1 mechanism shape: (32, 9)
M0 mechanism shape: (90, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (32, 90)
All JS distances: [0.405426771192577, 0.32937410423810237, 0.4187110726888325, 0.3303100660099741, 0.405426771192577, 0.32937410423810237, 0.41871107268883256, 0.33031006600997415, 0.40542677119257703, 0.32937410423810237, 0.4187110726888325, 0.33031006600997415]

Abstraction error: 0.41871107268883256

M1: ['X', 'Y'] -> ['Z', 'W', 'V', 'U']
M0: ['A', 'B'] -> ['D', 'E', 'F', 'G', 'H', 'I']
M1 mechanism shape: (768, 9)
M0 mechanism shape: (2160, 12)
Alpha_s shape: (9, 12)
Alpha_t shape: (768, 2160)
All JS distances: [0.39973021767551487, 0.32992196427396286, 0.4526503892321359, 0.33792044926318304, 0.40058892543929475, 0.3213555287340874, 0.4478572914674657, 0.3327225588243928, 0.39698050333465595, 0.3280283350230562, 0.4497822086638764, 0.33257820528191384]

Abstraction error: 0.4526503892321359

M1: ['X', 'W'] -> ['Z']
M0: ['A', 'E'] -> ['D', 'H']
M1 mechanism shape: (24, 6)
M0 mechanism shape: (24, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (24, 24)
All JS distances: [0.14671651019565898, 0.14671651019565896, 0.1467165101956589, 0.12190811938213303, 0.12190811938213311, 0.12190811938213304, 0.12808385511874865, 0.12808385511874854, 0.12808385511874867]

Abstraction error: 0.14671651019565898

M1: ['X', 'W'] -> ['Z', 'Y']
M0: ['A', 'E'] -> ['B', 'D', 'H']
M1 mechanism shape: (72, 6)
M0 mechanism shape: (96, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (72, 96)
All JS distances: [0.17608426608658084, 0.17608426608658082, 0.17608426608658082, 0.16899071049739622, 0.16899071049739622, 0.1689907104973962, 0.16789680207707058, 0.16789680207707058, 0.1678968020770706]

Abstraction error: 0.17608426608658084

M1: ['X', 'W'] -> ['Z', 'V']
M0: ['A', 'E'] -> ['D', 'F', 'H', 'I']
M1 mechanism shape: (192, 6)
M0 mechanism shape: (360, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (192, 360)
All JS distances: [0.19444751237104732, 0.19444751237104732, 0.19444751237104738, 0.17935359708817675, 0.1793535970881767, 0.17935359708817672, 0.18275350770860507, 0.1827535077086051, 0.1827535077086051]

Abstraction error: 0.19444751237104738

M1: ['X', 'W'] -> ['Z', 'U']
M0: ['A', 'E'] -> ['D', 'G', 'H']
M1 mechanism shape: (48, 6)
M0 mechanism shape: (48, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (48, 48)
All JS distances: [0.2769616321616567, 0.27696163216165665, 0.2769616321616567, 0.26613411163933676, 0.26613411163933676, 0.26613411163933676, 0.2700857185711583, 0.27008571857115826, 0.2700857185711583]

Abstraction error: 0.2769616321616567

M1: ['X', 'W'] -> ['Z', 'Y', 'V']
M0: ['A', 'E'] -> ['B', 'D', 'F', 'H', 'I']
M1 mechanism shape: (576, 6)
M0 mechanism shape: (1440, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (576, 1440)
All JS distances: [0.28110674920375944, 0.2811067492037594, 0.28110674920375944, 0.2720271820224432, 0.2720271820224432, 0.2720271820224432, 0.2731384357167785, 0.2731384357167784, 0.2731384357167785]

Abstraction error: 0.28110674920375944

M1: ['X', 'W'] -> ['Z', 'Y', 'U']
M0: ['A', 'E'] -> ['B', 'D', 'G', 'H']
M1 mechanism shape: (144, 6)
M0 mechanism shape: (192, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (144, 192)
All JS distances: [0.2985561473694319, 0.298556147369432, 0.298556147369432, 0.29025665191045436, 0.29025665191045436, 0.29025665191045436, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759]

Abstraction error: 0.298556147369432

M1: ['X', 'W'] -> ['Z', 'V', 'U']
M0: ['A', 'E'] -> ['D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (384, 6)
M0 mechanism shape: (720, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (384, 720)
All JS distances: [0.29864578015620025, 0.29864578015620025, 0.29864578015620025, 0.29179360402924703, 0.29179360402924703, 0.29179360402924703, 0.2929775776422845, 0.2929775776422845, 0.2929775776422845]

Abstraction error: 0.29864578015620025

M1: ['X', 'W'] -> ['Z', 'Y', 'V', 'U']
M0: ['A', 'E'] -> ['B', 'D', 'F', 'G', 'H', 'I']
M1 mechanism shape: (1152, 6)
M0 mechanism shape: (2880, 9)
Alpha_s shape: (6, 9)
Alpha_t shape: (1152, 2880)
All JS distances: [0.35028191225451477, 0.3502819122545147, 0.35028191225451477, 0.34679695557911117, 0.34679695557911117, 0.34679695557911117, 0.347648699875281, 0.347648699875281, 0.347648699875281]

Abstraction error: 0.35028191225451477

M1: ['X', 'V'] -> ['Z']
M0: ['A', 'F', 'I'] -> ['D', 'H']
M1 mechanism shape: (24, 24)
M0 mechanism shape: (24, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (24, 24)
All JS distances: [0.14671651019565893, 0.14671651019565898, 0.14671651019565896, 0.14671651019565898, 0.14671651019565893, 0.14671651019565893, 0.14671651019565898, 0.14671651019565893, 0.14671651019565896, 0.14671651019565898, 0.14671651019565893, 0.146716510195659, 0.14671651019565896, 0.14671651019565893, 0.14671651019565898, 0.12190811938213302, 0.121908119382133, 0.12190811938213293, 0.12190811938213303, 0.12190811938213303, 0.12190811938213302, 0.12190811938213303, 0.12190811938213283, 0.12190811938213317, 0.12190811938213304, 0.12190811938213297, 0.12190811938213296, 0.12190811938213293, 0.12190811938213303, 0.1219081193821329, 0.1280838551187487, 0.12808385511874867, 0.12808385511874862, 0.12808385511874862, 0.12808385511874862, 0.1280838551187486, 0.1280838551187487, 0.12808385511874867, 0.12808385511874862, 0.12808385511874867, 0.12808385511874862, 0.12808385511874862, 0.12808385511874865, 0.12808385511874867, 0.1280838551187487]

Abstraction error: 0.146716510195659

M1: ['X', 'V'] -> ['Z', 'Y']
M0: ['A', 'F', 'I'] -> ['B', 'D', 'H']
M1 mechanism shape: (72, 24)
M0 mechanism shape: (96, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (72, 96)
All JS distances: [0.17608426608658082, 0.17608426608658087, 0.17608426608658082, 0.17608426608658084, 0.17608426608658087, 0.17608426608658082, 0.17608426608658082, 0.17608426608658087, 0.1760842660865808, 0.17608426608658087, 0.17608426608658087, 0.17608426608658087, 0.17608426608658082, 0.17608426608658084, 0.17608426608658082, 0.1689907104973962, 0.16899071049739625, 0.16899071049739625, 0.1689907104973962, 0.1689907104973962, 0.16899071049739622, 0.1689907104973962, 0.1689907104973962, 0.16899071049739622, 0.1689907104973962, 0.16899071049739625, 0.16899071049739622, 0.16899071049739625, 0.16899071049739622, 0.16899071049739622, 0.16789680207707058, 0.16789680207707058, 0.16789680207707058, 0.1678968020770706, 0.16789680207707058, 0.1678968020770706, 0.16789680207707058, 0.16789680207707058, 0.1678968020770706, 0.16789680207707058, 0.1678968020770706, 0.16789680207707056, 0.16789680207707058, 0.16789680207707058, 0.16789680207707058]

Abstraction error: 0.17608426608658087

M1: ['X', 'V'] -> ['Z', 'W']
M0: ['A', 'F', 'I'] -> ['D', 'E', 'H']
M1 mechanism shape: (48, 24)
M0 mechanism shape: (72, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (48, 72)
All JS distances: [0.24813107454223934, 0.24813107454223937, 0.24813107454223926, 0.2481310745422393, 0.24813107454223934, 0.24813107454223932, 0.24813107454223934, 0.24813107454223932, 0.24813107454223934, 0.2481310745422393, 0.24813107454223934, 0.24813107454223932, 0.24813107454223934, 0.24813107454223934, 0.2481310745422393, 0.25725674622460787, 0.2572567462246079, 0.2572567462246078, 0.25725674622460787, 0.25725674622460787, 0.25725674622460787, 0.2572567462246079, 0.25725674622460787, 0.25725674622460787, 0.2572567462246079, 0.2572567462246078, 0.25725674622460787, 0.2572567462246079, 0.2572567462246079, 0.25725674622460787, 0.2634095747826151, 0.26340957478261506, 0.2634095747826152, 0.2634095747826151, 0.2634095747826151, 0.2634095747826151, 0.2634095747826152, 0.26340957478261506, 0.2634095747826152, 0.2634095747826151, 0.2634095747826151, 0.2634095747826151, 0.2634095747826151, 0.2634095747826151, 0.2634095747826151]

Abstraction error: 0.2634095747826152

M1: ['X', 'V'] -> ['Z', 'U']
M0: ['A', 'F', 'I'] -> ['D', 'G', 'H']
M1 mechanism shape: (48, 24)
M0 mechanism shape: (48, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (48, 48)
All JS distances: [0.2769616321616567, 0.27696163216165676, 0.27696163216165665, 0.2769616321616567, 0.2769616321616567, 0.27696163216165665, 0.27696163216165665, 0.27696163216165665, 0.2769616321616567, 0.2769616321616567, 0.2769616321616567, 0.27696163216165665, 0.2769616321616567, 0.2769616321616567, 0.2769616321616567, 0.26613411163933676, 0.26613411163933676, 0.26613411163933676, 0.2661341116393368, 0.26613411163933676, 0.26613411163933676, 0.2661341116393367, 0.26613411163933676, 0.26613411163933676, 0.26613411163933676, 0.26613411163933676, 0.2661341116393367, 0.26613411163933676, 0.26613411163933676, 0.2661341116393367, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.27008571857115826, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583, 0.2700857185711583]

Abstraction error: 0.27696163216165676

M1: ['X', 'V'] -> ['Z', 'Y', 'W']
M0: ['A', 'F', 'I'] -> ['B', 'D', 'E', 'H']
M1 mechanism shape: (144, 24)
M0 mechanism shape: (288, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (144, 288)
All JS distances: [0.3427808206762295, 0.34278082067622956, 0.34278082067622956, 0.3427808206762295, 0.34278082067622956, 0.34278082067622956, 0.3427808206762295, 0.3427808206762295, 0.3427808206762295, 0.3427808206762295, 0.3427808206762295, 0.3427808206762295, 0.34278082067622956, 0.3427808206762295, 0.34278082067622956, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.3541977353658485, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.35419773536584853, 0.3552252450340687, 0.3552252450340687, 0.3552252450340688, 0.3552252450340687, 0.3552252450340687, 0.3552252450340688, 0.3552252450340687, 0.3552252450340687, 0.3552252450340687, 0.35522524503406877, 0.3552252450340688, 0.35522524503406877, 0.3552252450340687, 0.3552252450340687, 0.35522524503406877]

Abstraction error: 0.3552252450340688

M1: ['X', 'V'] -> ['Z', 'Y', 'U']
M0: ['A', 'F', 'I'] -> ['B', 'D', 'G', 'H']
M1 mechanism shape: (144, 24)
M0 mechanism shape: (192, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (144, 192)
All JS distances: [0.298556147369432, 0.298556147369432, 0.298556147369432, 0.298556147369432, 0.298556147369432, 0.298556147369432, 0.298556147369432, 0.2985561473694319, 0.298556147369432, 0.298556147369432, 0.298556147369432, 0.2985561473694319, 0.298556147369432, 0.298556147369432, 0.298556147369432, 0.2902566519104544, 0.2902566519104544, 0.29025665191045436, 0.2902566519104544, 0.2902566519104544, 0.2902566519104544, 0.2902566519104544, 0.29025665191045436, 0.29025665191045436, 0.2902566519104544, 0.2902566519104544, 0.29025665191045436, 0.2902566519104544, 0.2902566519104544, 0.2902566519104544, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.2947506608063759, 0.29475066080637596, 0.2947506608063759, 0.29475066080637596]

Abstraction error: 0.298556147369432

M1: ['X', 'V'] -> ['Z', 'W', 'U']
M0: ['A', 'F', 'I'] -> ['D', 'E', 'G', 'H']
M1 mechanism shape: (96, 24)
M0 mechanism shape: (144, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (96, 144)
All JS distances: [0.34968192148698235, 0.34968192148698235, 0.3496819214869823, 0.34968192148698235, 0.34968192148698235, 0.34968192148698235, 0.34968192148698235, 0.34968192148698235, 0.34968192148698235, 0.3496819214869824, 0.3496819214869823, 0.34968192148698235, 0.3496819214869823, 0.34968192148698235, 0.34968192148698235, 0.34135716378151104, 0.3413571637815111, 0.3413571637815111, 0.3413571637815111, 0.3413571637815111, 0.3413571637815111, 0.3413571637815111, 0.34135716378151104, 0.3413571637815111, 0.3413571637815111, 0.3413571637815111, 0.3413571637815111, 0.34135716378151104, 0.34135716378151104, 0.3413571637815111, 0.34625050188033957, 0.3462505018803395, 0.3462505018803395, 0.3462505018803396, 0.3462505018803396, 0.3462505018803396, 0.34625050188033957, 0.3462505018803396, 0.34625050188033957, 0.34625050188033957, 0.3462505018803396, 0.3462505018803396, 0.34625050188033957, 0.34625050188033957, 0.34625050188033957]

Abstraction error: 0.3496819214869824

M1: ['X', 'V'] -> ['Z', 'Y', 'W', 'U']
M0: ['A', 'F', 'I'] -> ['B', 'D', 'E', 'G', 'H']
M1 mechanism shape: (288, 24)
M0 mechanism shape: (576, 45)
Alpha_s shape: (24, 45)
Alpha_t shape: (288, 576)
All JS distances: [0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41382556346877014, 0.41000991689621746, 0.41000991689621746, 0.4100099168962174, 0.41000991689621746, 0.41000991689621746, 0.41000991689621746, 0.41000991689621746, 0.4100099168962174, 0.41000991689621746, 0.41000991689621746, 0.41000991689621746, 0.4100099168962174, 0.41000991689621746, 0.4100099168962174, 0.4100099168962174, 0.413002665790578, 0.41300266579057804, 0.41300266579057804, 0.413002665790578, 0.41300266579057804, 0.413002665790578, 0.4130026657905781, 0.41300266579057804, 0.413002665790578, 0.413002665790578, 0.4130026657905781, 0.41300266579057804, 0.41300266579057804, 0.41300266579057804, 0.41300266579057804]

Abstraction error: 0.41382556346877014

M1: ['X', 'U'] -> ['Z']
M0: ['A', 'G'] -> ['D', 'H']
M1 mechanism shape: (24, 6)
M0 mechanism shape: (24, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (24, 24)
All JS distances: [0.14671651019565898, 0.14671651019565887, 0.12190811938213302, 0.12190811938213296, 0.12808385511874867, 0.12808385511874873]

Abstraction error: 0.14671651019565898

M1: ['X', 'U'] -> ['W']
M0: ['A', 'G'] -> ['E']
M1 mechanism shape: (2, 6)
M0 mechanism shape: (3, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (2, 3)
All JS distances: [0.28365370223347247, 0.28282402813485197, 0.28365370223347247, 0.28282402813485186, 0.28365370223347247, 0.28282402813485186]

Abstraction error: 0.28365370223347247

M1: ['X', 'U'] -> ['V']
M0: ['A', 'G'] -> ['F', 'I']
M1 mechanism shape: (8, 6)
M0 mechanism shape: (15, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (8, 15)
All JS distances: [0.12491355211742403, 0.12508085160141333, 0.12491355211742408, 0.12508085160141316, 0.12491355211742415, 0.12508085160141336]

Abstraction error: 0.12508085160141336

M1: ['X', 'U'] -> ['Z', 'Y']
M0: ['A', 'G'] -> ['B', 'D', 'H']
M1 mechanism shape: (72, 6)
M0 mechanism shape: (96, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (72, 96)
All JS distances: [0.17608426608658087, 0.17608426608658082, 0.16899071049739617, 0.16899071049739617, 0.16789680207707064, 0.16789680207707056]

Abstraction error: 0.17608426608658087

M1: ['X', 'U'] -> ['Z', 'W']
M0: ['A', 'G'] -> ['D', 'E', 'H']
M1 mechanism shape: (48, 6)
M0 mechanism shape: (72, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (48, 72)
All JS distances: [0.2487407301129614, 0.2471985979532554, 0.25704910159709954, 0.2578838613013556, 0.2634662693319804, 0.2634656709845331]

Abstraction error: 0.2634662693319804

M1: ['X', 'U'] -> ['Z', 'V']
M0: ['A', 'G'] -> ['D', 'F', 'H', 'I']
M1 mechanism shape: (192, 6)
M0 mechanism shape: (360, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (192, 360)
All JS distances: [0.1941390150598212, 0.19539733305082624, 0.17793216270040604, 0.18255656209991944, 0.18194585025204763, 0.18464478506138912]

Abstraction error: 0.19539733305082624

M1: ['X', 'U'] -> ['Y', 'W']
M0: ['A', 'G'] -> ['B', 'E']
M1 mechanism shape: (6, 6)
M0 mechanism shape: (12, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (6, 12)
All JS distances: [0.3822774086698545, 0.37921353595613094, 0.38227740866985455, 0.3792135359561309, 0.3822774086698545, 0.3792135359561309]

Abstraction error: 0.38227740866985455

M1: ['X', 'U'] -> ['Y', 'V']
M0: ['A', 'G'] -> ['B', 'F', 'I']
M1 mechanism shape: (24, 6)
M0 mechanism shape: (60, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (24, 60)
All JS distances: [0.254750211201766, 0.25607436349513335, 0.254750211201766, 0.2560743634951334, 0.25475021120176594, 0.2560743634951334]

Abstraction error: 0.2560743634951334

M1: ['X', 'U'] -> ['W', 'V']
M0: ['A', 'G'] -> ['E', 'F', 'I']
M1 mechanism shape: (16, 6)
M0 mechanism shape: (45, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (16, 45)
All JS distances: [0.3156586091271352, 0.3145615145859717, 0.31565860912713517, 0.3145615145859717, 0.31565860912713517, 0.3145615145859717]

Abstraction error: 0.3156586091271352

M1: ['X', 'U'] -> ['Z', 'Y', 'W']
M0: ['A', 'G'] -> ['B', 'D', 'E', 'H']
M1 mechanism shape: (144, 6)
M0 mechanism shape: (288, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (144, 288)
All JS distances: [0.3432884805446631, 0.342005111274922, 0.35422418753182844, 0.3544217578451525, 0.35529623405803906, 0.35527777913605035]

Abstraction error: 0.35529623405803906

M1: ['X', 'U'] -> ['Z', 'Y', 'V']
M0: ['A', 'G'] -> ['B', 'D', 'F', 'H', 'I']
M1 mechanism shape: (576, 6)
M0 mechanism shape: (1440, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (576, 1440)
All JS distances: [0.280867833979249, 0.2821296039708743, 0.2712336944254569, 0.27422099191597266, 0.2715413990392777, 0.27669219342839985]

Abstraction error: 0.2821296039708743

M1: ['X', 'U'] -> ['Z', 'W', 'V']
M0: ['A', 'G'] -> ['D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (384, 6)
M0 mechanism shape: (1080, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (384, 1080)
All JS distances: [0.31958979884175076, 0.3159265957641189, 0.3108271950811182, 0.3093360144361637, 0.3138420500764171, 0.31121871047548305]

Abstraction error: 0.31958979884175076

M1: ['X', 'U'] -> ['Y', 'W', 'V']
M0: ['A', 'G'] -> ['B', 'E', 'F', 'I']
M1 mechanism shape: (48, 6)
M0 mechanism shape: (180, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (48, 180)
All JS distances: [0.4227121132118712, 0.4203751677620154, 0.42271211321187113, 0.4203751677620154, 0.42271211321187113, 0.4203751677620154]

Abstraction error: 0.4227121132118712

M1: ['X', 'U'] -> ['Z', 'Y', 'W', 'V']
M0: ['A', 'G'] -> ['B', 'D', 'E', 'F', 'H', 'I']
M1 mechanism shape: (1152, 6)
M0 mechanism shape: (4320, 6)
Alpha_s shape: (6, 6)
Alpha_t shape: (1152, 4320)
All JS distances: [0.4195494038543588, 0.41066182990035927, 0.41455707931866315, 0.40716935308345226, 0.4154378225305543, 0.40764810714187083]

Abstraction error: 0.4195494038543588

M1: ['Z', 'Y'] -> ['W']
M0: ['B', 'D', 'H'] -> ['E']
M1 mechanism shape: (2, 72)
M0 mechanism shape: (3, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (2, 3)
All JS distances: [0.3243736203962745, 0.15211218969749027, 0.32737726890087365, 0.20772528694790407, 0.3243736203962745, 0.15211218969749082, 0.3273772689008734, 0.20772528694790407, 0.3243736203962745, 0.1521121896974908, 0.3273772689008734, 0.20772528694790407, 0.3243736203962745, 0.15211218969749082, 0.32737726890087354, 0.2077252869479041, 0.3243736203962744, 0.15211218969749057, 0.32737726890087365, 0.20772528694790407, 0.3243736203962742, 0.15211218969749055, 0.3273772689008734, 0.20772528694790407, 0.34484411570226187, 0.1735419811534569, 0.34782680469637967, 0.22889733085472044, 0.3448441157022619, 0.17354198115345695, 0.3478268046963797, 0.22889733085472028, 0.34484411570226187, 0.17354198115345687, 0.3478268046963797, 0.22889733085472025, 0.3448441157022619, 0.17354198115345687, 0.34782680469637955, 0.2288973308547203, 0.344844115702262, 0.17354198115345698, 0.3478268046963797, 0.22889733085472044, 0.3448441157022619, 0.17354198115345693, 0.34782680469637955, 0.2288973308547202, 0.33645042135049913, 0.164743036105472, 0.3394418600185987, 0.2202085088659417, 0.33645042135049924, 0.16474303610547175, 0.3394418600185987, 0.22020850886594198, 0.33645042135049924, 0.1647430361054718, 0.3394418600185986, 0.22020850886594198, 0.3364504213504991, 0.16474303610547175, 0.3394418600185986, 0.2202085088659417, 0.33645042135049924, 0.16474303610547175, 0.33944186001859883, 0.22020850886594198, 0.33645042135049896, 0.1647430361054718, 0.3394418600185988, 0.22020850886594193, 0.3286886073035678, 0.15662127315532776, 0.33168794390767475, 0.21218306043526683, 0.32868860730356786, 0.1566212731553278, 0.33168794390767475, 0.21218306043526702, 0.32868860730356775, 0.15662127315532767, 0.33168794390767475, 0.2121830604352672, 0.3286886073035679, 0.15662127315532784, 0.33168794390767475, 0.21218306043526716, 0.3286886073035677, 0.15662127315532734, 0.33168794390767475, 0.21218306043526694, 0.3286886073035678, 0.15662127315532787, 0.33168794390767464, 0.21218306043526702]

Abstraction error: 0.3478268046963797

M1: ['Z', 'Y'] -> ['V']
M0: ['B', 'D', 'H'] -> ['F', 'I']
M1 mechanism shape: (8, 72)
M0 mechanism shape: (15, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (8, 15)
All JS distances: [0.12059895477285086, 0.16461649482620247, 0.12441147613953613, 0.0920804934336397, 0.12059895477285075, 0.16461649482620255, 0.12441147613953614, 0.09208049343363983, 0.12059895477285086, 0.16461649482620244, 0.12441147613953622, 0.09208049343363993, 0.12059895477285079, 0.16461649482620244, 0.12441147613953633, 0.09208049343363972, 0.12059895477285068, 0.16461649482620255, 0.12441147613953622, 0.09208049343363997, 0.12059895477285078, 0.1646164948262024, 0.12441147613953629, 0.09208049343363983, 0.12024251924615571, 0.16748634037318316, 0.12414767918008246, 0.09374331046460979, 0.12024251924615553, 0.16748634037318338, 0.12414767918008233, 0.09374331046460974, 0.12024251924615578, 0.16748634037318336, 0.12414767918008225, 0.09374331046460962, 0.12024251924615569, 0.16748634037318333, 0.12414767918008233, 0.09374331046460974, 0.12024251924615577, 0.1674863403731833, 0.12414767918008243, 0.09374331046460957, 0.12024251924615567, 0.16748634037318333, 0.12414767918008245, 0.09374331046460972, 0.11803412367187929, 0.16790291026041623, 0.12175151889575893, 0.09580872957869215, 0.11803412367187914, 0.1679029102604162, 0.1217515188957591, 0.09580872957869203, 0.11803412367187899, 0.16790291026041626, 0.1217515188957589, 0.09580872957869224, 0.11803412367187892, 0.16790291026041615, 0.12175151889575912, 0.09580872957869245, 0.11803412367187918, 0.16790291026041626, 0.12175151889575904, 0.09580872957869231, 0.11803412367187911, 0.16790291026041626, 0.12175151889575886, 0.09580872957869203, 0.1190756500862472, 0.15925877106784023, 0.1224311609345235, 0.09189169585012594, 0.11907565008624736, 0.1592587710678404, 0.1224311609345237, 0.09189169585012594, 0.11907565008624735, 0.15925877106784023, 0.12243116093452361, 0.09189169585012585, 0.11907565008624732, 0.15925877106784014, 0.12243116093452353, 0.09189169585012591, 0.11907565008624739, 0.15925877106784034, 0.1224311609345235, 0.0918916958501257, 0.11907565008624742, 0.1592587710678402, 0.12243116093452347, 0.09189169585012572]

Abstraction error: 0.16790291026041626

M1: ['Z', 'Y'] -> ['X', 'W']
M0: ['B', 'D', 'H'] -> ['A', 'E']
M1 mechanism shape: (6, 72)
M0 mechanism shape: (9, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (6, 9)
All JS distances: [0.40880533656407525, 0.3111127812492301, 0.41086388309195004, 0.33744839849582076, 0.4088053365640753, 0.31111278124923, 0.4108638830919501, 0.33744839849582076, 0.4088053365640752, 0.3111127812492301, 0.41086388309195004, 0.3374483984958207, 0.40880533656407525, 0.31111278124923003, 0.41086388309195004, 0.33744839849582065, 0.40880533656407536, 0.3111127812492301, 0.4108638830919501, 0.3374483984958207, 0.4088053365640753, 0.31111278124923, 0.41086388309195004, 0.3374483984958208, 0.422947855199684, 0.3205291540735976, 0.4250498107939696, 0.34894798983015657, 0.422947855199684, 0.3205291540735976, 0.4250498107939695, 0.34894798983015657, 0.422947855199684, 0.3205291540735976, 0.42504981079396953, 0.3489479898301565, 0.422947855199684, 0.3205291540735976, 0.4250498107939696, 0.3489479898301565, 0.422947855199684, 0.3205291540735976, 0.4250498107939696, 0.3489479898301565, 0.4229478551996839, 0.3205291540735976, 0.42504981079396953, 0.34894798983015646, 0.4171005460171852, 0.3165465187590903, 0.4191855782204413, 0.34413943545955067, 0.4171005460171852, 0.3165465187590903, 0.4191855782204413, 0.3441394354595507, 0.4171005460171852, 0.31654651875909035, 0.41918557822044117, 0.34413943545955067, 0.41710054601718527, 0.3165465187590903, 0.4191855782204413, 0.3441394354595508, 0.4171005460171852, 0.3165465187590903, 0.4191855782204413, 0.3441394354595507, 0.4171005460171852, 0.3165465187590903, 0.4191855782204413, 0.3441394354595507, 0.4117528416077283, 0.3130133706897362, 0.41382114723966096, 0.339807615261772, 0.41175284160772824, 0.3130133706897363, 0.41382114723966096, 0.33980761526177194, 0.41175284160772824, 0.31301337068973645, 0.4138211472396609, 0.33980761526177206, 0.4117528416077283, 0.3130133706897364, 0.41382114723966085, 0.339807615261772, 0.4117528416077283, 0.3130133706897363, 0.4138211472396609, 0.339807615261772, 0.4117528416077283, 0.3130133706897363, 0.4138211472396609, 0.33980761526177194]

Abstraction error: 0.4250498107939696

M1: ['Z', 'Y'] -> ['X', 'V']
M0: ['B', 'D', 'H'] -> ['A', 'F', 'I']
M1 mechanism shape: (24, 72)
M0 mechanism shape: (45, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (24, 45)
All JS distances: [0.2991638243761056, 0.31677035332580605, 0.3005007770124829, 0.2903711668984961, 0.2991638243761056, 0.31677035332580605, 0.30050077701248296, 0.29037116689849607, 0.29916382437610567, 0.31677035332580605, 0.300500777012483, 0.2903711668984961, 0.2991638243761056, 0.3167703533258061, 0.30050077701248296, 0.2903711668984961, 0.2991638243761056, 0.31677035332580605, 0.30050077701248296, 0.2903711668984961, 0.2991638243761056, 0.31677035332580605, 0.3005007770124829, 0.2903711668984961, 0.2990368205866147, 0.3180725737541079, 0.3004024130243228, 0.29082471552319605, 0.2990368205866146, 0.318072573754108, 0.3004024130243228, 0.290824715523196, 0.2990368205866147, 0.31807257375410797, 0.30040241302432275, 0.29082471552319605, 0.2990368205866146, 0.31807257375410797, 0.30040241302432275, 0.29082471552319594, 0.2990368205866146, 0.3180725737541079, 0.3004024130243228, 0.290824715523196, 0.2990368205866147, 0.31807257375410797, 0.30040241302432275, 0.29082471552319594, 0.298287302372281, 0.3182336614953352, 0.2995660011370453, 0.29139751463660996, 0.29828730237228107, 0.3182336614953353, 0.29956600113704535, 0.29139751463660996, 0.2982873023722811, 0.3182336614953352, 0.29956600113704535, 0.29139751463660996, 0.29828730237228107, 0.3182336614953352, 0.29956600113704535, 0.29139751463661, 0.2982873023722811, 0.3182336614953352, 0.2995660011370453, 0.29139751463660996, 0.29828730237228107, 0.3182336614953353, 0.2995660011370453, 0.29139751463661, 0.29865810647493135, 0.31435753418546325, 0.2998223450847281, 0.29032357801221265, 0.29865810647493135, 0.3143575341854633, 0.29982234508472816, 0.2903235780122126, 0.2986581064749313, 0.31435753418546325, 0.29982234508472816, 0.2903235780122127, 0.29865810647493135, 0.31435753418546325, 0.2998223450847282, 0.2903235780122126, 0.29865810647493135, 0.31435753418546325, 0.29982234508472816, 0.2903235780122126, 0.29865810647493135, 0.3143575341854633, 0.29982234508472816, 0.2903235780122126]

Abstraction error: 0.3182336614953353

M1: ['Z', 'Y'] -> ['W', 'V']
M0: ['B', 'D', 'H'] -> ['E', 'F', 'I']
M1 mechanism shape: (16, 72)
M0 mechanism shape: (45, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (16, 45)
All JS distances: [0.34912516804539734, 0.22182611592893187, 0.35328020564033563, 0.23846745662288427, 0.3491251680453973, 0.221826115928932, 0.3532802056403356, 0.2384674566228843, 0.3491251680453974, 0.221826115928932, 0.35328020564033563, 0.23846745662288435, 0.34912516804539734, 0.2218261159289319, 0.35328020564033563, 0.2384674566228843, 0.3491251680453974, 0.22182611592893198, 0.3532802056403356, 0.23846745662288438, 0.34912516804539734, 0.2218261159289319, 0.3532802056403356, 0.23846745662288435, 0.36786398829704264, 0.2378987812948967, 0.37197809045499175, 0.25775306695920785, 0.3678639882970427, 0.23789878129489664, 0.3719780904549917, 0.25775306695920797, 0.3678639882970427, 0.23789878129489667, 0.3719780904549918, 0.2577530669592079, 0.3678639882970427, 0.23789878129489664, 0.37197809045499175, 0.257753066959208, 0.3678639882970427, 0.2378987812948967, 0.37197809045499175, 0.2577530669592079, 0.36786398829704275, 0.2378987812948967, 0.3719780904549917, 0.25775306695920797, 0.36011403371253203, 0.23245205890598, 0.3641392077843358, 0.2515371348569704, 0.36011403371253203, 0.23245205890598, 0.3641392077843359, 0.2515371348569704, 0.36011403371253203, 0.23245205890597995, 0.3641392077843359, 0.2515371348569704, 0.36011403371253203, 0.23245205890598, 0.36413920778433584, 0.2515371348569704, 0.36011403371253203, 0.23245205890598, 0.3641392077843359, 0.2515371348569704, 0.36011403371253203, 0.23245205890597997, 0.36413920778433584, 0.25153713485697027, 0.35246399095523406, 0.22107126033470448, 0.35643335103761986, 0.24240421566679024, 0.35246399095523406, 0.22107126033470442, 0.3564333510376199, 0.2424042156667902, 0.352463990955234, 0.2210712603347045, 0.35643335103761986, 0.2424042156667902, 0.35246399095523406, 0.22107126033470456, 0.35643335103761986, 0.24240421566679027, 0.35246399095523406, 0.22107126033470445, 0.35643335103761986, 0.24240421566679027, 0.35246399095523406, 0.22107126033470442, 0.35643335103761986, 0.24240421566679013]

Abstraction error: 0.3719780904549918

M1: ['Z', 'Y'] -> ['W', 'U']
M0: ['B', 'D', 'H'] -> ['E', 'G']
M1 mechanism shape: (4, 72)
M0 mechanism shape: (6, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (4, 6)
All JS distances: [0.39438320471460553, 0.28744410140961674, 0.3965858523638051, 0.31685098948759366, 0.3943832047146057, 0.2874441014096168, 0.39658585236380517, 0.3168509894875937, 0.39438320471460553, 0.2874441014096168, 0.39658585236380517, 0.31685098948759355, 0.39438320471460553, 0.28744410140961674, 0.39658585236380517, 0.31685098948759366, 0.39438320471460553, 0.2874441014096169, 0.3965858523638051, 0.3168509894875935, 0.39438320471460575, 0.2874441014096169, 0.3965858523638052, 0.31685098948759366, 0.4084821481717384, 0.29724192732214655, 0.41071940592929074, 0.32863657532604384, 0.40848214817173845, 0.2972419273221464, 0.4107194059292909, 0.32863657532604373, 0.4084821481717384, 0.2972419273221463, 0.4107194059292909, 0.32863657532604384, 0.4084821481717383, 0.2972419273221464, 0.4107194059292908, 0.32863657532604384, 0.40848214817173845, 0.2972419273221465, 0.4107194059292908, 0.32863657532604384, 0.40848214817173845, 0.2972419273221463, 0.4107194059292908, 0.32863657532604384, 0.40202977286935604, 0.2926611143985863, 0.404251788877858, 0.3231905998367968, 0.40202977286935593, 0.2926611143985864, 0.404251788877858, 0.32319059983679693, 0.402029772869356, 0.2926611143985862, 0.404251788877858, 0.32319059983679677, 0.402029772869356, 0.2926611143985863, 0.404251788877858, 0.3231905998367969, 0.402029772869356, 0.29266111439858644, 0.40425178887785795, 0.3231905998367969, 0.402029772869356, 0.2926611143985864, 0.40425178887785806, 0.3231905998367969, 0.3967331072317267, 0.28901066872747594, 0.3989418529466808, 0.31878006309943846, 0.3967331072317268, 0.28901066872747583, 0.3989418529466807, 0.3187800630994386, 0.3967331072317268, 0.28901066872747594, 0.3989418529466807, 0.3187800630994384, 0.39673310723172667, 0.28901066872747583, 0.39894185294668083, 0.31878006309943846, 0.39673310723172667, 0.28901066872747594, 0.39894185294668083, 0.3187800630994385, 0.3967331072317268, 0.289010668727476, 0.3989418529466808, 0.3187800630994386]

Abstraction error: 0.4107194059292909

M1: ['Z', 'Y'] -> ['V', 'U']
M0: ['B', 'D', 'H'] -> ['F', 'G', 'I']
M1 mechanism shape: (16, 72)
M0 mechanism shape: (30, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (16, 30)
All JS distances: [0.25859365518313704, 0.2860417639995478, 0.2603185226142081, 0.25136522716498905, 0.258593655183137, 0.28604176399954784, 0.2603185226142081, 0.251365227164989, 0.258593655183137, 0.2860417639995479, 0.2603185226142081, 0.25136522716498905, 0.25859365518313704, 0.2860417639995478, 0.2603185226142083, 0.251365227164989, 0.2585936551831371, 0.28604176399954795, 0.2603185226142081, 0.25136522716498905, 0.2585936551831371, 0.28604176399954784, 0.2603185226142082, 0.251365227164989, 0.25867990572253974, 0.2879569193657261, 0.2604521666996601, 0.25222765547373277, 0.2586799057225398, 0.2879569193657259, 0.26045216669966004, 0.2522276554737328, 0.25867990572253974, 0.28795691936572604, 0.2604521666996601, 0.2522276554737328, 0.2586799057225397, 0.287956919365726, 0.2604521666996601, 0.2522276554737328, 0.25867990572253974, 0.2879569193657259, 0.26045216669966015, 0.25222765547373277, 0.2586799057225398, 0.287956919365726, 0.26045216669966004, 0.2522276554737328, 0.2603228201300665, 0.2884746603524752, 0.2622080500395168, 0.25254416836240134, 0.2603228201300665, 0.2884746603524752, 0.2622080500395168, 0.2525441683624014, 0.2603228201300666, 0.2884746603524752, 0.26220805003951686, 0.25254416836240146, 0.2603228201300665, 0.28847466035247515, 0.26220805003951686, 0.2525441683624014, 0.26032282013006647, 0.28847466035247527, 0.26220805003951686, 0.25254416836240146, 0.2603228201300665, 0.2884746603524753, 0.2622080500395169, 0.2525441683624014, 0.26710308259731463, 0.2825483928120622, 0.2686398426675365, 0.2542800363833759, 0.26710308259731463, 0.2825483928120622, 0.2686398426675365, 0.25428003638337576, 0.26710308259731463, 0.2825483928120622, 0.2686398426675365, 0.2542800363833757, 0.26710308259731463, 0.28254839281206223, 0.26863984266753654, 0.2542800363833757, 0.2671030825973146, 0.2825483928120623, 0.2686398426675365, 0.2542800363833758, 0.26710308259731463, 0.2825483928120623, 0.26863984266753654, 0.2542800363833757]

Abstraction error: 0.2884746603524753

M1: ['Z', 'Y'] -> ['X', 'W', 'V']
M0: ['B', 'D', 'H'] -> ['A', 'E', 'F', 'I']
M1 mechanism shape: (48, 72)
M0 mechanism shape: (135, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (48, 135)
All JS distances: [0.42636337837605315, 0.34534262938368604, 0.4293566393313273, 0.3546027536098169, 0.42636337837605315, 0.3453426293836861, 0.4293566393313273, 0.3546027536098169, 0.42636337837605315, 0.34534262938368604, 0.4293566393313273, 0.35460275360981697, 0.42636337837605315, 0.34534262938368604, 0.42935663933132734, 0.35460275360981697, 0.4263633783760531, 0.3453426293836861, 0.42935663933132734, 0.3546027536098169, 0.42636337837605315, 0.3453426293836861, 0.4293566393313273, 0.35460275360981686, 0.4397519325059378, 0.3544161636639346, 0.442781554547398, 0.3659966320708905, 0.4397519325059378, 0.3544161636639346, 0.442781554547398, 0.3659966320708905, 0.4397519325059378, 0.3544161636639346, 0.442781554547398, 0.3659966320708905, 0.4397519325059378, 0.3544161636639346, 0.4427815545473979, 0.3659966320708905, 0.4397519325059378, 0.3544161636639345, 0.442781554547398, 0.36599663207089056, 0.4397519325059378, 0.3544161636639346, 0.4427815545473979, 0.36599663207089045, 0.43416934376610145, 0.35128470662191663, 0.43710601002863136, 0.3622661795207941, 0.43416934376610145, 0.35128470662191663, 0.4371060100286313, 0.3622661795207941, 0.43416934376610145, 0.3512847066219166, 0.4371060100286313, 0.3622661795207941, 0.4341693437661014, 0.3512847066219166, 0.4371060100286313, 0.3622661795207941, 0.43416934376610145, 0.3512847066219166, 0.4371060100286313, 0.3622661795207941, 0.43416934376610145, 0.3512847066219166, 0.43710601002863136, 0.3622661795207941, 0.4286827442762165, 0.34487662756946746, 0.4315509357003252, 0.356858474762501, 0.42868274427621655, 0.34487662756946746, 0.4315509357003252, 0.3568584747625011, 0.42868274427621655, 0.34487662756946746, 0.4315509357003252, 0.35685847476250104, 0.4286827442762165, 0.3448766275694675, 0.43155093570032516, 0.35685847476250104, 0.4286827442762165, 0.3448766275694674, 0.4315509357003252, 0.35685847476250104, 0.42868274427621655, 0.34487662756946746, 0.4315509357003252, 0.35685847476250104]

Abstraction error: 0.442781554547398

M1: ['Z', 'Y'] -> ['X', 'W', 'U']
M0: ['B', 'D', 'H'] -> ['A', 'E', 'G']
M1 mechanism shape: (12, 72)
M0 mechanism shape: (18, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (12, 18)
All JS distances: [0.45960721510988506, 0.38457206213520384, 0.4612560455139111, 0.4041070780080405, 0.45960721510988506, 0.3845720621352039, 0.4612560455139109, 0.4041070780080405, 0.45960721510988506, 0.3845720621352039, 0.461256045513911, 0.4041070780080405, 0.459607215109885, 0.3845720621352039, 0.461256045513911, 0.4041070780080406, 0.459607215109885, 0.38457206213520384, 0.461256045513911, 0.4041070780080404, 0.45960721510988506, 0.38457206213520384, 0.4612560455139111, 0.4041070780080404, 0.47019222432399815, 0.39097437651142114, 0.4718884600613193, 0.4121889161880714, 0.4701922243239981, 0.39097437651142114, 0.47188846006131924, 0.41218891618807135, 0.4701922243239981, 0.3909743765114212, 0.47188846006131924, 0.4121889161880714, 0.47019222432399815, 0.39097437651142114, 0.4718884600613193, 0.4121889161880714, 0.47019222432399815, 0.39097437651142114, 0.4718884600613192, 0.4121889161880714, 0.4701922243239981, 0.39097437651142114, 0.47188846006131924, 0.4121889161880714, 0.4653327137867957, 0.38797015796993994, 0.46700780903891737, 0.4084393107882799, 0.4653327137867957, 0.38797015796993994, 0.4670078090389173, 0.40843931078827994, 0.4653327137867958, 0.38797015796994, 0.4670078090389173, 0.4084393107882799, 0.46533271378679575, 0.38797015796993994, 0.46700780903891737, 0.40843931078827994, 0.46533271378679564, 0.38797015796993994, 0.46700780903891737, 0.4084393107882799, 0.46533271378679575, 0.38797015796993994, 0.46700780903891737, 0.4084393107882799, 0.46136393013941684, 0.38559110398875185, 0.4630209763304938, 0.40542267479202243, 0.4613639301394169, 0.38559110398875185, 0.4630209763304938, 0.4054226747920224, 0.46136393013941684, 0.3855911039887518, 0.4630209763304938, 0.4054226747920224, 0.46136393013941684, 0.38559110398875185, 0.4630209763304939, 0.40542267479202243, 0.46136393013941684, 0.38559110398875185, 0.46302097633049377, 0.40542267479202243, 0.46136393013941684, 0.38559110398875185, 0.4630209763304939, 0.40542267479202243]

Abstraction error: 0.4718884600613193

M1: ['Z', 'Y'] -> ['X', 'V', 'U']
M0: ['B', 'D', 'H'] -> ['A', 'F', 'G', 'I']
M1 mechanism shape: (48, 72)
M0 mechanism shape: (90, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (48, 90)
All JS distances: [0.3664818767069689, 0.38380063830755273, 0.36754035949010233, 0.36208983893162316, 0.3664818767069689, 0.3838006383075527, 0.36754035949010244, 0.36208983893162316, 0.3664818767069689, 0.38380063830755273, 0.36754035949010244, 0.3620898389316231, 0.3664818767069689, 0.38380063830755273, 0.36754035949010233, 0.36208983893162316, 0.3664818767069689, 0.3838006383075527, 0.3675403594901024, 0.36208983893162316, 0.3664818767069689, 0.38380063830755273, 0.3675403594901024, 0.36208983893162316, 0.3665325891006172, 0.3850431423662941, 0.36762007427313126, 0.3626074149466041, 0.36653258910061726, 0.38504314236629406, 0.36762007427313126, 0.36260741494660403, 0.3665325891006172, 0.38504314236629406, 0.36762007427313126, 0.3626074149466041, 0.3665325891006172, 0.3850431423662941, 0.3676200742731312, 0.3626074149466041, 0.36653258910061726, 0.3850431423662941, 0.36762007427313126, 0.3626074149466041, 0.3665325891006172, 0.38504314236629406, 0.3676200742731312, 0.36260741494660403, 0.3675768867377368, 0.3853969751815981, 0.36873918647064496, 0.3628264884232426, 0.3675768867377368, 0.3853969751815981, 0.36873918647064496, 0.3628264884232426, 0.3675768867377368, 0.3853969751815981, 0.3687391864706449, 0.3628264884232426, 0.3675768867377368, 0.38539697518159804, 0.36873918647064496, 0.3628264884232426, 0.3675768867377368, 0.38539697518159804, 0.36873918647064496, 0.36282648842324267, 0.36757688673773675, 0.3853969751815981, 0.36873918647064496, 0.3628264884232426, 0.3717343559725993, 0.38155178659369016, 0.37269621755691984, 0.3638491898654, 0.3717343559725993, 0.38155178659369016, 0.3726962175569199, 0.36384918986539994, 0.37173435597259924, 0.38155178659369027, 0.3726962175569199, 0.3638491898654, 0.3717343559725993, 0.3815517865936902, 0.3726962175569199, 0.36384918986539994, 0.3717343559725993, 0.38155178659369016, 0.37269621755691984, 0.3638491898654, 0.3717343559725993, 0.38155178659369016, 0.37269621755691984, 0.36384918986539994]

Abstraction error: 0.3853969751815981

M1: ['Z', 'Y'] -> ['W', 'V', 'U']
M0: ['B', 'D', 'H'] -> ['E', 'F', 'G', 'I']
M1 mechanism shape: (32, 72)
M0 mechanism shape: (90, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (32, 90)
All JS distances: [0.4054267711925769, 0.3188759737785627, 0.4087103041084735, 0.3272228922787054, 0.40542677119257703, 0.3188759737785627, 0.4087103041084735, 0.3272228922787054, 0.40542677119257703, 0.31887597377856264, 0.4087103041084735, 0.3272228922787054, 0.40542677119257703, 0.3188759737785627, 0.4087103041084735, 0.3272228922787054, 0.405426771192577, 0.3188759737785627, 0.4087103041084735, 0.3272228922787054, 0.405426771192577, 0.31887597377856264, 0.4087103041084734, 0.3272228922787054, 0.4205839967496819, 0.3293741042381023, 0.4238782621119741, 0.34064180297839264, 0.4205839967496819, 0.3293741042381024, 0.42387826211197405, 0.34064180297839264, 0.4205839967496819, 0.3293741042381023, 0.42387826211197405, 0.34064180297839264, 0.42058399674968183, 0.32937410423810237, 0.4238782621119741, 0.3406418029783926, 0.4205839967496819, 0.32937410423810237, 0.42387826211197405, 0.3406418029783926, 0.4205839967496819, 0.3293741042381023, 0.42387826211197405, 0.3406418029783926, 0.4153445760858343, 0.32593287278558847, 0.41871107268883256, 0.3355839353054377, 0.4153445760858343, 0.3259328727855884, 0.4187110726888325, 0.3355839353054377, 0.4153445760858343, 0.3259328727855884, 0.4187110726888325, 0.33558393530543773, 0.4153445760858343, 0.3259328727855884, 0.41871107268883256, 0.3355839353054377, 0.41534457608583436, 0.3259328727855884, 0.4187110726888325, 0.33558393530543773, 0.4153445760858343, 0.3259328727855884, 0.4187110726888325, 0.3355839353054377, 0.4111830515894335, 0.3173275538327636, 0.4144106213677353, 0.33031006600997415, 0.4111830515894335, 0.3173275538327636, 0.4144106213677353, 0.3303100660099741, 0.4111830515894335, 0.3173275538327637, 0.4144106213677353, 0.3303100660099741, 0.4111830515894335, 0.3173275538327636, 0.4144106213677354, 0.3303100660099741, 0.4111830515894335, 0.3173275538327637, 0.4144106213677353, 0.3303100660099741, 0.4111830515894335, 0.3173275538327636, 0.4144106213677353, 0.3303100660099741]

Abstraction error: 0.4238782621119741

M1: ['Z', 'Y'] -> ['X', 'W', 'V', 'U']
M0: ['B', 'D', 'H'] -> ['A', 'E', 'F', 'G', 'I']
M1 mechanism shape: (96, 72)
M0 mechanism shape: (270, 96)
Alpha_s shape: (72, 96)
Alpha_t shape: (96, 270)
All JS distances: [0.4680222241240685, 0.4056231689675544, 0.47053074221833496, 0.41129959384402504, 0.46802222412406846, 0.40562316896755446, 0.47053074221833496, 0.41129959384402504, 0.4680222241240686, 0.4056231689675544, 0.47053074221833496, 0.41129959384402504, 0.4680222241240685, 0.40562316896755446, 0.47053074221833496, 0.41129959384402504, 0.4680222241240685, 0.4056231689675544, 0.47053074221833496, 0.41129959384402515, 0.4680222241240685, 0.4056231689675544, 0.47053074221833496, 0.4112995938440251, 0.47952999864099455, 0.4128462190587115, 0.4820795922943742, 0.4206479151341468, 0.47952999864099455, 0.4128462190587115, 0.4820795922943742, 0.42064791513414673, 0.47952999864099455, 0.4128462190587115, 0.4820795922943742, 0.4206479151341468, 0.47952999864099455, 0.4128462190587115, 0.4820795922943742, 0.4206479151341468, 0.4795299986409945, 0.41284621905871144, 0.4820795922943742, 0.42064791513414673, 0.47952999864099455, 0.4128462190587115, 0.4820795922943742, 0.4206479151341468, 0.4755397774805603, 0.4104758458139255, 0.4781360723756702, 0.41710633971861727, 0.4755397774805604, 0.4104758458139255, 0.47813607237567024, 0.41710633971861727, 0.4755397774805604, 0.41047584581392543, 0.4781360723756702, 0.41710633971861727, 0.4755397774805603, 0.4104758458139255, 0.47813607237567024, 0.41710633971861727, 0.4755397774805603, 0.4104758458139255, 0.47813607237567024, 0.4171063397186172, 0.4755397774805603, 0.4104758458139255, 0.4781360723756702, 0.41710633971861727, 0.4723774342729483, 0.4045626303217239, 0.47485669437565464, 0.4134293189802729, 0.4723774342729483, 0.4045626303217239, 0.4748566943756547, 0.4134293189802729, 0.4723774342729483, 0.40456263032172385, 0.4748566943756547, 0.41342931898027296, 0.4723774342729483, 0.4045626303217239, 0.4748566943756547, 0.4134293189802729, 0.47237743427294826, 0.40456263032172396, 0.4748566943756547, 0.4134293189802729, 0.4723774342729483, 0.4045626303217239, 0.4748566943756547, 0.4134293189802729]

Abstraction error: 0.4820795922943742

M1: ['Z', 'U'] -> ['W']
M0: ['D', 'G', 'H'] -> ['E']
M1 mechanism shape: (2, 48)
M0 mechanism shape: (3, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (2, 3)
All JS distances: [0.2836537022334726, 0.28365370223347247, 0.28365370223347264, 0.2836537022334725, 0.2836537022334725, 0.28365370223347247, 0.28282402813485197, 0.2828240281348519, 0.28282402813485186, 0.28282402813485186, 0.28282402813485186, 0.2828240281348519, 0.28365370223347247, 0.28365370223347247, 0.28365370223347247, 0.2836537022334726, 0.28365370223347247, 0.28365370223347247, 0.282824028134852, 0.2828240281348518, 0.28282402813485186, 0.28282402813485197, 0.28282402813485163, 0.2828240281348519, 0.2836537022334725, 0.2836537022334726, 0.28365370223347264, 0.2836537022334725, 0.2836537022334724, 0.28365370223347247, 0.2828240281348518, 0.28282402813485186, 0.28282402813485186, 0.2828240281348518, 0.2828240281348518, 0.2828240281348519, 0.2836537022334725, 0.2836537022334725, 0.28365370223347264, 0.2836537022334725, 0.28365370223347247, 0.28365370223347247, 0.28282402813485197, 0.28282402813485186, 0.2828240281348519, 0.282824028134852, 0.28282402813485186, 0.28282402813485197]

Abstraction error: 0.28365370223347264

M1: ['Z', 'U'] -> ['V']
M0: ['D', 'G', 'H'] -> ['F', 'I']
M1 mechanism shape: (8, 48)
M0 mechanism shape: (15, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (8, 15)
All JS distances: [0.12491355211742426, 0.12491355211742422, 0.12491355211742432, 0.12491355211742418, 0.12491355211742415, 0.12491355211742428, 0.12508085160141325, 0.12508085160141333, 0.1250808516014133, 0.12508085160141336, 0.12508085160141333, 0.1250808516014133, 0.12491355211742415, 0.12491355211742411, 0.12491355211742416, 0.12491355211742408, 0.12491355211742411, 0.12491355211742416, 0.12508085160141322, 0.12508085160141325, 0.12508085160141333, 0.12508085160141338, 0.12508085160141327, 0.1250808516014134, 0.12491355211742425, 0.1249135521174243, 0.12491355211742435, 0.12491355211742415, 0.12491355211742412, 0.12491355211742437, 0.12508085160141333, 0.1250808516014133, 0.12508085160141325, 0.12508085160141333, 0.12508085160141325, 0.12508085160141316, 0.12491355211742422, 0.1249135521174241, 0.12491355211742408, 0.12491355211742414, 0.12491355211742411, 0.1249135521174243, 0.12508085160141333, 0.12508085160141336, 0.1250808516014131, 0.1250808516014134, 0.125080851601413, 0.1250808516014133]

Abstraction error: 0.1250808516014134

M1: ['Z', 'U'] -> ['X', 'W']
M0: ['D', 'G', 'H'] -> ['A', 'E']
M1 mechanism shape: (6, 48)
M0 mechanism shape: (9, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (6, 9)
All JS distances: [0.3818566742398929, 0.38185667423989283, 0.38185667423989283, 0.38185667423989283, 0.38185667423989283, 0.38185667423989283, 0.3813293431200948, 0.3813293431200948, 0.3813293431200948, 0.38132934312009475, 0.3813293431200948, 0.38132934312009475, 0.3818566742398927, 0.3818566742398929, 0.3818566742398928, 0.38185667423989283, 0.38185667423989283, 0.38185667423989283, 0.3813293431200948, 0.3813293431200947, 0.3813293431200948, 0.38132934312009487, 0.38132934312009487, 0.38132934312009475, 0.38185667423989283, 0.38185667423989283, 0.3818566742398929, 0.38185667423989283, 0.38185667423989283, 0.3818566742398928, 0.3813293431200947, 0.38132934312009475, 0.38132934312009475, 0.38132934312009475, 0.3813293431200948, 0.38132934312009487, 0.38185667423989283, 0.3818566742398928, 0.38185667423989283, 0.38185667423989283, 0.38185667423989283, 0.38185667423989283, 0.38132934312009475, 0.38132934312009487, 0.38132934312009475, 0.38132934312009487, 0.38132934312009475, 0.38132934312009487]

Abstraction error: 0.3818566742398929

M1: ['Z', 'U'] -> ['X', 'V']
M0: ['D', 'G', 'H'] -> ['A', 'F', 'I']
M1 mechanism shape: (24, 48)
M0 mechanism shape: (45, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (24, 45)
All JS distances: [0.30068382098935564, 0.3006838209893557, 0.30068382098935564, 0.30068382098935564, 0.30068382098935564, 0.3006838209893557, 0.30074920378752945, 0.30074920378752945, 0.30074920378752945, 0.3007492037875294, 0.3007492037875294, 0.3007492037875294, 0.3006838209893557, 0.30068382098935564, 0.30068382098935564, 0.3006838209893557, 0.30068382098935564, 0.30068382098935564, 0.30074920378752945, 0.30074920378752945, 0.3007492037875294, 0.3007492037875294, 0.3007492037875294, 0.30074920378752945, 0.30068382098935564, 0.30068382098935564, 0.3006838209893557, 0.30068382098935575, 0.3006838209893557, 0.30068382098935575, 0.30074920378752945, 0.30074920378752945, 0.30074920378752945, 0.30074920378752945, 0.30074920378752945, 0.30074920378752945, 0.3006838209893557, 0.30068382098935564, 0.30068382098935575, 0.3006838209893557, 0.3006838209893557, 0.3006838209893557, 0.30074920378752945, 0.30074920378752945, 0.30074920378752945, 0.3007492037875294, 0.30074920378752945, 0.3007492037875294]

Abstraction error: 0.30074920378752945

M1: ['Z', 'U'] -> ['Y', 'W']
M0: ['D', 'G', 'H'] -> ['B', 'E']
M1 mechanism shape: (6, 48)
M0 mechanism shape: (12, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (6, 12)
All JS distances: [0.3822774086698545, 0.38227740866985455, 0.3822774086698545, 0.38227740866985455, 0.3822774086698545, 0.3822774086698545, 0.3792135359561309, 0.379213535956131, 0.37921353595613094, 0.3792135359561309, 0.37921353595613083, 0.379213535956131, 0.3822774086698545, 0.38227740866985455, 0.38227740866985455, 0.38227740866985443, 0.3822774086698545, 0.3822774086698545, 0.3792135359561309, 0.3792135359561309, 0.37921353595613094, 0.3792135359561309, 0.3792135359561309, 0.379213535956131, 0.38227740866985455, 0.3822774086698545, 0.38227740866985455, 0.38227740866985443, 0.3822774086698545, 0.3822774086698545, 0.37921353595613094, 0.37921353595613083, 0.3792135359561309, 0.3792135359561309, 0.37921353595613094, 0.379213535956131, 0.3822774086698545, 0.38227740866985455, 0.3822774086698545, 0.3822774086698545, 0.38227740866985455, 0.3822774086698545, 0.37921353595613094, 0.3792135359561309, 0.3792135359561309, 0.379213535956131, 0.3792135359561309, 0.37921353595613094]

Abstraction error: 0.38227740866985455

M1: ['Z', 'U'] -> ['Y', 'V']
M0: ['D', 'G', 'H'] -> ['B', 'F', 'I']
M1 mechanism shape: (24, 48)
M0 mechanism shape: (60, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (24, 60)
All JS distances: [0.254750211201766, 0.25475021120176605, 0.25475021120176594, 0.25475021120176594, 0.25475021120176594, 0.25475021120176605, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.25607436349513346, 0.2560743634951334, 0.25475021120176594, 0.254750211201766, 0.25475021120176594, 0.25475021120176594, 0.25475021120176594, 0.25475021120176594, 0.25607436349513346, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.25475021120176605, 0.254750211201766, 0.254750211201766, 0.25475021120176594, 0.254750211201766, 0.254750211201766, 0.2560743634951334, 0.25607436349513335, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334, 0.254750211201766, 0.254750211201766, 0.254750211201766, 0.254750211201766, 0.25475021120176594, 0.25475021120176594, 0.2560743634951334, 0.2560743634951334, 0.25607436349513335, 0.2560743634951334, 0.2560743634951334, 0.2560743634951334]

Abstraction error: 0.25607436349513346

M1: ['Z', 'U'] -> ['W', 'V']
M0: ['D', 'G', 'H'] -> ['E', 'F', 'I']
M1 mechanism shape: (16, 48)
M0 mechanism shape: (45, 48)
Alpha_s shape: (48, 48)
Alpha_t shape: (16, 45)
All JS distances: [0.31565860912713517, 0.31565860912713517, 0.31565860912713506, 0.31565860912713506, 0.3156586091271351, 0.31565860912713506, 0.3145615145859717, 0.3145615145859717, 0.3145615145859717, 0.3145615145859717, 0.3145615145859717, 0.3145615145859717, 0.3156586091271351, 0.3156586091271351, 0.3156586091271352, 0.3156586091271351, 0.31565860912713517, 0.3156586091271351, 0.3145615145859717, 0.31456151458597165, 0.31456151458597165, 0.3145615145859717, 0.31456151458597176, 0.3145615145859717, 0.3156586091271351, 0.3156586091271351, 0.31565860912713517, 0.3156586091271351, 0.31565860912713517, 0.31565860912713517, 0.31456151458597165, 0.3145615145859717, 0.31456151458597165, 0.31456151458597165, 0.3145615145859717, 0.31456151458597176, 0.3156586091271351, 0.3156586091271351, 0.31565860912713517, 0.31565860912713517, 0.