Structural comparison metrics

class pyagrum.StructuralMetrics

Compares two graphs of the same type and size, computing precision, recall, F-score, SHD and SID metrics at both the skeleton level (ignoring orientation) and the arc/edge level (including orientation).

After calling compare(), the confusion-matrix accessors (tp(), fp(), fn(), tn() and their _skeleton variants) become available, as do all derived metrics.

Examples

>>> import pyagrum as gum
>>> ref = pyagrum.fastBN('A->B->C;A->C')
>>> test = pyagrum.fastBN('A->B;C->B;A->C')
>>> comp = pyagrum.StructuralMetrics()
>>> comp.compare(ref, test)
>>> comp.precision()
>>> comp.sid(ref, test)

StructuralMetrics() -> StructuralMetrics : default constructor

compare(*args)

Compare two graphs of the same type and equal size, populating the internal confusion matrix used by all metric accessors.

The following overloads are available:

• compare(DiGraph, DiGraph) — nodes matched by NodeId (no alignment).
• compare(UndiGraph, UndiGraph) — nodes matched by NodeId (no alignment).
• compare(PDAG, PDAG) — nodes matched by NodeId (no alignment).
• compare(BayesNet, BayesNet) — nodes matched by variable name; comparison runs on the corresponding essential graphs (CPDAGs).
• compare(BayesNet, PDAG) — ref BN is converted to its essential graph first.
• compare(PDAG, BayesNet) — test BN is converted to its essential graph first.

For a DAG-level comparison of two BayesNets, align NodeIds manually (e.g. by building an aligned BN by variable name) and call compare(DiGraph, DiGraph) with bn.dag(). SID (see sid()) is the only metric that natively operates on the DAGs of BayesNets.

• Parameters:
  • ref – Reference graph (DiGraph, UndiGraph, PDAG, or BayesNet).
  • test – Graph to evaluate against the reference (same type as ref, or PDAG when ref is a BayesNet and vice-versa).
• Raises:
  • pyagrum.OperationNotAllowed – If both graphs are empty, or if their sizes differ.
  • pyagrum.InvalidNode – If a node present in ref is absent from test (for BayesNet overloads, matching is done by variable name).
• Return type: None

f_score()

F1-score at the arc/edge level: harmonic mean of precision and recall.

• Returns: F1-score in [0, 1].
• Return type: float

f_score_skeleton()

F1-score at the skeleton level: harmonic mean of skeleton precision and recall.

• Returns: Skeleton F1-score in [0, 1].
• Return type: float

fn()

Number of false negatives at the arc/edge level (arcs/edges present in ref but missing from test).

• Returns: False-negative count.
• Return type: float

fn_skeleton()

Number of false negatives at the skeleton level (undirected edges present in the ref skeleton but missing from the test skeleton).

• Returns: False-negative count for the skeleton.
• Return type: float

fp()

Number of false positives at the arc/edge level (extra or misoriented arcs/edges in the test graph).

• Returns: False-positive count.
• Return type: float

fp_skeleton()

Number of false positives at the skeleton level (extra undirected edges in the test skeleton).

• Returns: False-positive count for the skeleton.
• Return type: float

precision()

Precision at the arc/edge level (orientation included): TP / (TP + FP).

• Returns: Precision in [0, 1].
• Return type: float

precision_skeleton()

Precision at the skeleton level: TP / (TP + FP).

• Returns: Skeleton precision in [0, 1].
• Return type: float

recall()

Recall at the arc/edge level (orientation included): TP / (TP + FN).

• Returns: Recall in [0, 1].
• Return type: float

recall_skeleton()

Recall at the skeleton level: TP / (TP + FN).

• Returns: Skeleton recall in [0, 1].
• Return type: float

shd()

Structural Hamming Distance: number of arc/edge insertions, deletions and reversals needed to transform the test graph into the ref graph.

• Returns: SHD (non-negative integer returned as float).
• Return type: float

shd_skeleton()

Structural Hamming Distance at the skeleton level: number of edge insertions and deletions needed to transform the test skeleton into the ref skeleton.

• Returns: Skeleton SHD (non-negative integer returned as float).
• Return type: float

sid(*args)

Compute the Structural Intervention Distance (SID) between two DAGs (Peters & Bühlmann, 2015).

SID counts the number of ordered pairs (i, j) for which the parent-adjustment set for the causal effect of i on j in the reference graph is incorrectly reproduced by the test graph.

Properties:

• sid(G, G) == 0
• If G ⊆ H (all parents of G are parents of H), then sid(G, H) == 0.
• SID is asymmetric: sid(G, H) != sid(H, G) in general.

Two overloads are available:

• sid(DAG, DAG) — operates directly on DAG objects.

• sid(BayesNet, BayesNet) — nodes matched by variable name; the comparison runs on the underlying DAGs after name-based alignment.

• Parameters:

  • ref – Reference DAG or BayesNet.
  • test – DAG or BayesNet to evaluate against the reference.

• Returns: The SID value (non-negative integer returned as float).

• Return type: float

• Raises:

  • pyagrum.OperationNotAllowed – If the graphs have different sizes.
  • pyagrum.InvalidNode – If a variable present in ref is absent from test (BayesNet overload only).

tn()

Number of true negatives at the arc/edge level (pairs correctly absent from both ref and test).

• Returns: True-negative count.
• Return type: float

tn_skeleton()

Number of true negatives at the skeleton level (pairs correctly absent from both ref and test skeletons).

• Returns: True-negative count for the skeleton.
• Return type: float

tp()

Number of true positives at the arc/edge level (correctly predicted arcs or edges, orientation included for directed graphs).

• Returns: True-positive count.
• Return type: float

tp_skeleton()

Number of true positives at the skeleton level (undirected edges correctly predicted, ignoring orientation).

• Returns: True-positive count for the skeleton.
• Return type: float

SEE ALSO

pyagrum.lib.bn_vs_bn.GraphicalBNComparator : a higher-level, Bayesian-network-oriented wrapper around pyagrum.StructuralMetrics (see Comparison of Bayesian networks).