Skip to content

CausalIQ Analysis Metrics

This module provides functions for analysing and comparing causal graphs, including structural metrics, Kullback-Leibler divergence calculations, and Bayesys compatibility metrics.

Core Functions

pdag_compare 

pdag_compare(
    graph: Any,
    reference: Any,
    bayesys: Optional[str] = None,
    identify_edges: bool = False,
) -> Dict[str, Any]

Compare a pdag with a reference pdag.

Parameters:

  • graph

    (PDAG) –

    graph which is to be compared

  • reference

    (PDAG) –

    reference graph for comparison

  • bayesys

    (str, default: None ) –

    version of Bayesys metrics to return, or None if not required

  • identify_edges

    (bool, default: False ) –

    whether edges in each low level category (e.g. arc_missing) are to be included in metrics returned.

Raises:

  • TypeError –

    if bad argument types

Returns:

  • dict ( Dict[str, Any] ) –

    structural comparison metrics

Compare a PDAG with a reference PDAG to compute structural comparison metrics including precision, recall, F1 score, and Structural Hamming Distance (SHD).

kl 

kl(dist: Series, ref_dist: Series) -> float

Returns the Kullback-Liebler Divergence of dist from ref_dist.

Parameters:

  • dist

    (Series) –

    distribution to compute KL from ...

  • ref_dist

    (Series) –

    ... the reference/theoretical distribution

Raises:

  • TypeError –

    if both arguments not Panda Series

  • ValueError –

    if dists have different indices or bad values

Returns:

  • float ( float ) –

    divergence value

Compute the Kullback-Leibler divergence of one probability distribution from another reference distribution.

bayesys_metrics 

bayesys_metrics(
    metrics: Dict[str, Union[int, float, None]],
    max_edges: int,
    num_ref_edges: int,
    version: str,
) -> Dict[str, float]

Compute Bayesys-compatible metrics from structural comparison results, including precision, recall, F1 with half-match support, Bayesys Scoring Function (BSF), and Delta Dependency Measure (DDM).

Overview

PDAG Comparison

The pdag_compare function provides comprehensive structural comparison between two Partially Directed Acyclic Graphs (PDAGs). It computes detailed edge-level metrics including:

  • Arc metrics: matched, reversed, missing, extra
  • Edge metrics: matched, missing, extra, not-arc conversions
  • Summary metrics: precision, recall, F1 score, Structural Hamming Distance (SHD)
  • Bayesys compatibility: Optional Bayesys v1.3-v1.6 metrics

The function includes built-in sanity checks to ensure metric consistency and can optionally identify specific edges in each category for detailed analysis.

Distribution Analysis

The kl function computes Kullback-Leibler divergence for comparing probability distributions, commonly used in causal discovery for independence testing and model comparison.

Bayesys Metrics

The bayesys_metrics function converts structural metrics to Bayesys-compatible format, supporting the half-match concept where reversed arcs and edge/arc mismatches are treated as partial matches. This enables comparison with results from the Bayesys causal discovery software.

Usage Examples

Basic PDAG Comparison

from causaliq_analysis.metrics import pdag_compare
from causaliq_core.graph import PDAG

# Compare two PDAGs
result = pdag_compare(learned_graph, reference_graph)
print(f"F1 Score: {result['f1']}")
print(f"SHD: {result['shd']}")

With Bayesys Compatibility

# Include Bayesys v1.6 metrics
result = pdag_compare(learned_graph, reference_graph, bayesys="v1.6")
print(f"Bayesys F1: {result['f1-b']}")
print(f"BSF Score: {result['bsf']}")

KL Divergence

from causaliq_analysis.metrics import kl
import pandas as pd

# Compare two probability distributions
dist1 = pd.Series([0.4, 0.3, 0.3], index=['A', 'B', 'C'])
dist2 = pd.Series([0.33, 0.33, 0.34], index=['A', 'B', 'C'])
divergence = kl(dist1, dist2)
print(f"KL Divergence: {divergence}")