Explaining a model

import time

import pandas as pd

import pyagrum as gum
import pyagrum.lib.explain as expl

/var/folders/r1/pj4vdx_n4_d_xpsb04kzf97r0000gp/T/ipykernel_93107/453429602.py:6: DeprecationWarning: The module 'pyagrum.lib.explain' has been deprecated since version 2.2.2. Please use the 'pyagrum.explain' module instead.
  import pyagrum.lib.explain as expl

Building the model

We build a simple graph for the example

template = gum.fastBN("X1->X2->Y;X3->Z->Y;X0->Z;X1->Z;X2->R[5];Z->R;X1->Y")
data_path = "res/shap/Data_6var_direct_indirect.csv"

## gum.generateSample(template,1000,data_path)

learner = gum.BNLearner(data_path, template)
bn = learner.learnParameters(template.dag())
bn

1-independence list (w.r.t. the class Y)

Given a model, it may be interesting to investigate the conditional independences of the class Y created by this very model.

## this function explores all the CI between 2 variables and computes the p-values w.r.t to a csv file.
expl.independenceListForPairs(bn, data_path)

{('R', 'X0', ('X1', 'Z')): 0.7083382647903902,
 ('R', 'X1', ('X2', 'Z')): 0.4693848625409949,
 ('R', 'X3', ('X1', 'Z')): 0.4128522974536623,
 ('R', 'Y', ('X2', 'Z')): 0.8684231094674687,
 ('X0', 'X1', ()): 0.723302358657366,
 ('X0', 'X2', ()): 0.9801394906304377,
 ('X0', 'X3', ()): 0.7676868597218647,
 ('X0', 'Y', ('X1', 'Z')): 0.5816487109659612,
 ('X1', 'X3', ()): 0.5216508257424717,
 ('X2', 'X3', ()): 0.9837021981131505,
 ('X2', 'Z', ('X1',)): 0.6638491605436834,
 ('X3', 'Y', ('X1', 'Z')): 0.8774081450472305}

… with respect to a specific target.

expl.independenceListForPairs(bn, data_path, target="Y")

{('Y', 'R', ('X2', 'Z')): 0.8684231094674687,
 ('Y', 'X0', ('X1', 'Z')): 0.5816487109659612,
 ('Y', 'X3', ('X1', 'Z')): 0.8774081450472305}

2-ShapValues : explaining a Bayesian network as a classifier

print(expl.ShapValues.__doc__)

Class to compute Shapley values for a target variable in a Bayesian network.

The ShapValue class implements the calculation of Shap values in Bayesian networks. It is necessary to specify a target and to provide a Bayesian network whose parameters are known and will be used later in the different calculation methods.

gumshap = expl.ShapValues(bn, "Y")

Compute Conditionnal in Bayesian Network

A dataset (as a pandas.dataframe) must be provided so that the Bayesian network can learn its parameters and then predict.

The method conditional computes the conditonal shap values using the Bayesian Networks. It returns 2 graphs and a dictionary. The first one shows the distribution of the shap values for each of the variables, the second one classifies the variables by their importance.

train = pd.read_csv(data_path).sample(frac=1.0)

t_start = time.time()
resultat = gumshap.conditional(train, plot=True, plot_importance=True, percentage=False)
print(f"Run Time : {time.time() - t_start} sec")

Run Time : 11.231132745742798 sec

t_start = time.time()
resultat = gumshap.conditional(train, plot=False, plot_importance=True, percentage=False)
print(f"Run Time : {time.time() - t_start} sec")

Run Time : 11.17263412475586 sec

result = gumshap.conditional(train, plot=True, plot_importance=False, percentage=False)
## result is a Dict[str,float] of the different Shapley values for all nodes.

The result is returned as a dictionary, the keys are the names of the features and the associated value is the absolute value of the average of the calculated shap.

t_start = time.time()
resultat = gumshap.conditional(train, plot=False, plot_importance=False, percentage=False)
print(f"Run Time : {time.time() - t_start} sec")
resultat

Run Time : 11.124483823776245 sec





{'X2': 0.32716064437520076,
 'X1': 0.25333754053706536,
 'X0': 0.06176712200000174,
 'R': 0.05445633444152398,
 'X3': 0.10465402104047901,
 'Z': 0.5464180054433384}

Causal Shap Values

This method is similar to the previous one, except the formula of computation. It computes the causal shap value as described in the paper of Heskes Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models .

t_start = time.time()
causal = gumshap.causal(train, plot=True, plot_importance=True, percentage=False)
print(f"Run Time : {time.time() - t_start} sec")

Run Time : 197.2022829055786 sec

As you can see, since $R$ is not among the ‘causes’ of Y, its causal importance is null.

Marginal Shap Values

Similarly, one can also compute marginal Shap Value.

t_start = time.time()
marginal = gumshap.marginal(train, sample_size=10, plot=True, plot_importance=True, percentage=False)
print(f"Run Time : {time.time() - t_start} sec")
print(marginal)

Run Time : 1.130695104598999 sec
{'X2': 0.3496505738571567, 'X1': 0.32542485798597676, 'X0': 0.0, 'R': 0.0, 'X3': 0.0, 'Z': 0.7316965254273765}

As you can see, since $R$, $X0$ and $X3$ are not in the Markov Blanket of $Y$, their marginal importances are null.

Saving the graph

You can specify a filename if you prefer to save this figure instead of showing it:

t_start = time.time()
causal2 = gumshap.causal(train, plot=True, plot_importance=True, percentage=False, filename="out/marginal.pdf")
print(f"Run Time : {time.time() - t_start} sec")
print(causal2)

Run Time : 184.4341540336609 sec
{'X2': 0.2566212418283342, 'X1': 0.17251999198303453, 'X0': 0.09554457779312346, 'R': 0.14450002061335038, 'X3': 0.15489545419467637, 'Z': 0.466559465035866}

Visualizing shapvalues directly on a BN

This function returns a coloured graph that makes it easier to understand which variable is important and where it is located in the graph.

expl.showShapValues(bn, causal)

Visualizing information

Finally another view consists in showing the entropy on each node and the mutual informations on each arcs.

expl.showInformation(bn)