Skip to content

Evaluate Graph

evaluate_graph

The evaluate-graph command.

Functions:

evaluate_graph_cmd

evaluate_graph_cmd(
    input_graph: str,
    reference: str,
    metrics_requested: Tuple[str, ...],
    output: Optional[str],
    output_format: str,
) -> None

Evaluate a learned graph against a ground truth reference.

Computes structural accuracy metrics including F1 and SHD (Structural Hamming Distance). Supports both direct comparison and equivalence class comparison (comparing CPDAGs). Graphs may be deterministic (DAG/PDAG/CPDAG) or PDGs whose edge probabilities are compared fractionally; equivalence class metrics are unavailable for PDGs.

Supported metrics: - f1: F1 score from direct graph comparison - shd: Structural Hamming Distance from direct comparison - precision: Precision from direct comparison - recall: Recall from direct comparison - edge: Low-level edge counts from direct comparison (arc_matched, arc_reversed, edge_not_arc, arc_not_edge, edge_matched, arc_extra, edge_extra, arc_missing, edge_missing, missing_matched). Counts are fractional when either graph is a PDG. - equiv.f1: F1 score comparing equivalence classes (CPDAGs) - equiv.shd: SHD comparing equivalence classes (CPDAGs) - equiv.edge: Low-level edge counts comparing equivalence classes (CPDAGs)

Example

causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m f1 -m shd

causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m equiv.f1 -m equiv.shd

causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m edge -m equiv.edge

causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m f1 --format=table