Evaluate Graph¶
evaluate_graph
¶
The evaluate-graph command.
Functions:
-
evaluate_graph_cmd–Evaluate a learned graph against a ground truth reference.
evaluate_graph_cmd
¶
evaluate_graph_cmd(
input_graph: str,
reference: str,
metrics_requested: Tuple[str, ...],
output: Optional[str],
output_format: str,
) -> None
Evaluate a learned graph against a ground truth reference.
Computes structural accuracy metrics including F1 and SHD (Structural Hamming Distance). Supports both direct comparison and equivalence class comparison (comparing CPDAGs). Graphs may be deterministic (DAG/PDAG/CPDAG) or PDGs whose edge probabilities are compared fractionally; equivalence class metrics are unavailable for PDGs.
Supported metrics: - f1: F1 score from direct graph comparison - shd: Structural Hamming Distance from direct comparison - precision: Precision from direct comparison - recall: Recall from direct comparison - edge: Low-level edge counts from direct comparison (arc_matched, arc_reversed, edge_not_arc, arc_not_edge, edge_matched, arc_extra, edge_extra, arc_missing, edge_missing, missing_matched). Counts are fractional when either graph is a PDG. - equiv.f1: F1 score comparing equivalence classes (CPDAGs) - equiv.shd: SHD comparing equivalence classes (CPDAGs) - equiv.edge: Low-level edge counts comparing equivalence classes (CPDAGs)
Example
causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m f1 -m shd
causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m equiv.f1 -m equiv.shd
causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m edge -m equiv.edge
causaliq-analysis evaluate-graph -i learned.graphml \ -r ground_truth.graphml -m f1 --format=table