Summarisation Paradigm Design
Overview
The Summarisation Paradigm enables workflows to aggregate results from multiple matrix combinations into summary outputs. This is essential for causal discovery research where experiments are run across many parameter combinations (networks, sample sizes, algorithms) and results need to be combined for analysis, model averaging, or comparison.
Matrix Variable Handling
Matrix variables serve different purposes depending on the action pattern:
- CREATE pattern: Matrix variables must be used in
{{variable}}templates to produce distinct action invocations. - AGGREGATE pattern: Matrix variables define grouping dimensions for entry selection. They are not required in parameter templates since they control which cache entries are passed to the action.
- Run-once steps: A step writing a plain file runs once, so matrix variables do not define its dimensionality.
Motivation
Research Workflow Pattern
A typical causal discovery workflow generates many individual results:
matrix:
network: [asia, alarm, sachs]
sample_size: [100, 500, 1000]
seed: [1, 2, 3, 4, 5]
steps:
- name: Generate Graph
uses: causaliq/research
with:
action: generate_graph
output: results/graphs.db
This produces 45 individual graphs (3 × 3 × 5). Researchers then need to:
- Aggregate by network: Combine all graphs for each network
- Filter by quality: Only include successful runs
- Compute summaries: Average structures, consensus edges, etc.
The Summarisation Paradigm provides first-class support for this pattern.
Architecture
Aggregation Mode Detection
Aggregation mode is automatically activated when a step has:
- A workflow
matrixdefinition - An
inputparameter with.dbcache file(s)
def _is_aggregation_step(
self,
step: Dict[str, Any],
matrix: Dict[str, List[Any]],
) -> bool:
"""Check if a step should execute in aggregation mode."""
if not matrix:
return False
step_inputs = step.get("with", {})
input_param = step_inputs.get("input")
if input_param:
inputs = (
[input_param]
if isinstance(input_param, str)
else (list(input_param) if input_param else [])
)
return any(str(p).lower().endswith(".db") for p in inputs)
return False
Aggregation Configuration
The AggregationConfig dataclass captures all aggregation parameters:
@dataclass
class AggregationConfig:
input_caches: List[str] # Cache paths to scan
filter_expr: Optional[str] # Filter expression
matrix_vars: List[str] # Grouping dimensions
Execution Phases
Aggregation execution has two phases:
Phase 1: Scan
The scan phase reads input caches, applies filters, and groups entries:
def _scan_aggregation_inputs(
self,
config: AggregationConfig,
matrix_values: Dict[str, Any],
logger: Optional[Callable[[str], None]] = None,
) -> List[Dict[str, Any]]:
"""Scan input caches and collect entries matching matrix values."""
Scan Process:
The scan runs once per matrix job, but each input cache is read only once per run:
- Build or reuse a memoised scan index for each input cache. The
index holds each entry's parsed matrix values and flattened metadata,
read via
WorkflowCache.list_entry_summaries()without decompressing object payloads (e.g. graphs). - Apply the filter expression (if present) against the flattened
metadata; rejected entries are counted as
filtered. - Match surviving entries to the current matrix combination; non-matching entries are skipped.
- Load only the matched entries in full (via
cache.get()) and return them with provenance.
The random() filter population and the matching step both read from the
same index, so a large cache is not re-read for every matrix combination.
A cache's index is discarded whenever that cache is written (a step store
or an UPDATE), so later scans observe new entries.
Performance:
Scanning is O(cache entries) per input cache rather than O(matrix jobs × cache entries): the index is built once and reused across every matrix job and across the validation and execution passes. Only matched entries pay the cost of decompressing their object payloads.
Phase 2: Execute
The execute phase passes grouped entries to the action:
# In _execute_job:
agg_config = self._get_aggregation_config(step, matrix)
if agg_config is not None:
matching_entries = self._scan_aggregation_inputs(
agg_config,
job, # Current matrix values
)
resolved_inputs["_aggregation_entries"] = matching_entries
Actions receive entries via _aggregation_entries parameter.
Workflow Syntax
Basic Aggregation
matrix:
network: [asia, alarm]
steps:
# First step: generate individual graphs
- name: Generate
uses: causaliq/research
with:
action: generate_graph
output: graphs.db
# Second step: aggregate by network
- name: Summarise
uses: causaliq/analysis
with:
action: model_average
input: graphs.db
output: summaries.db
With Filter Expression
Filter entries before aggregation using matrix values, result-object fields and producer metadata:
- name: Summarise Successful
uses: causaliq/analysis
with:
action: model_average
input: graphs.db
filter: "learn_graph.status == 'ok' and learn_graph.edge_count > 0"
output: summaries.db
Multiple Input Caches
Combine results from multiple caches:
- name: Merge Results
uses: causaliq/analysis
with:
action: merge_graphs
input:
- pc_results.db
- ges_results.db
output: merged.db
Filter Expression Syntax
Filter expressions use a safe subset of Python syntax via simpleeval:
Supported Operators
| Operator | Example | Description |
|---|---|---|
== |
learn_graph.status == 'ok' |
Equality |
!= |
learn_graph.algorithm != 'pc' |
Inequality |
>, < |
learn_graph.edge_count > 5 |
Comparison |
>=, <= |
sample_size >= 100 |
Comparison |
and |
a > 5 and b < 10 |
Logical AND |
or |
a > 5 or b > 10 |
Logical OR |
not |
not failed |
Logical NOT |
in |
learn_graph.algorithm in ['pc', 'ges'] |
Membership |
Available Variables
Filter expressions can access:
- Matrix values:
network,sample_size, etc. - Result-object fields: bare names such as
f1orequiv.f1. - Metadata: producer-qualified names such as
learn_graph.statusorlearn_graph.hyperparameters.iss.
# Filter by matrix variable
filter: "sample_size >= 500"
# Filter by result-object field
filter: "f1 > 0.8"
# Filter by metadata field
filter: "learn_graph.status == 'ok'"
# Filter by qualified path
filter: "causaliq_research.generate_graph.node_count > 5"
Reporting View
Filters, summarise and plot evaluate one flat reporting view of each
entry:
- matrix values and result-object fields by bare name (
sample_size,f1); - metadata by producer-qualified name (
generate_graph.node_count,learn_graph.hyperparameters.iss); - explicit
generate_graph.parameters.*/generate_graph.result.*forms and provider-qualifiedcausaliq_research.generate_graph.node_countforms when needed.
Version 1 provider -> action -> fields metadata is normalised before
viewing. Provenance is recorded for reproducibility but is not exposed in
filters by default. Bare metadata names are rejected: the reference must
name the producer.
Entry Structure
Actions receive entries as dictionaries:
{
"matrix_values": {"network": "asia", "sample_size": 100},
"metadata": {...}, # Version 2 metadata envelope
"cache_path": "graphs.db",
"entry_hash": "abc123...",
"entry": CacheEntry(...) # Full entry object
}
Logging
Aggregation operations log statistics:
Aggregation scan: scanned=45, filtered=3, matched=15
scanned: Total entries examinedfiltered: Entries rejected by filtermatched: Entries matching current matrix values
Error Handling
Missing Caches
Non-existent caches are skipped with warning:
Warning: Cache does not exist: missing.db
Filter Errors
Filter evaluation errors cause entry to be filtered:
try:
if not evaluate_filter(filter_expr, flat_meta):
total_filtered += 1
continue
except Exception:
total_filtered += 1
continue
Cache Read Errors
Cache read failures are logged and skipped:
Warning: Failed to read cache graphs.db: database is locked
Integration with causaliq-core
The filter expression evaluation uses causaliq_core.utils:
from causaliq_core.utils import evaluate_filter
if config.filter_expr:
if not evaluate_filter(config.filter_expr, flat_meta):
continue
This provides consistent filter syntax across the CausalIQ ecosystem.
Future Enhancements
Weight-Based Aggregation
Support weighted combinations using compute_weight:
- name: Weighted Average
uses: causaliq/analysis
with:
action: model_average
aggregate: graphs.db
weight: "1.0 / sample_size" # Weight by inverse sample size
Hierarchical Aggregation
Multi-level aggregation for complex analyses:
- name: Per-Network Summary
with:
aggregate: graphs.db
group_by: [network]
- name: Global Summary
with:
aggregate: network_summaries.db
group_by: [] # Aggregate all