Core Concepts
CausalIQ Workflow uses a declarative YAML syntax inspired by GitHub Actions. This section explains the fundamental building blocks.
Workflows
A workflow is a YAML file defining a sequence of steps to execute. Every
workflow has a steps: section containing one or more steps.
# workflow.yml
steps:
- name: "Merge Graphs"
uses: "causaliq-analysis"
with:
action: "merge_graphs"
input: "results/graphs.db"
output: "results/merged.db"
Workflow Structure
| Property | Required | Description |
|---|---|---|
steps |
Yes | List of steps to execute |
matrix |
No | Parameter matrix for expansion (see below) |
Workflow Parameters
Any top-level key other than matrix and steps declares a workflow
parameter and its default value. A default may be a scalar, a list of
scalars, or null. Declare parameters before matrix and steps so the
intent is clear; the parser itself is order-independent. Declared
parameters are available as template variables in step parameters, and a
caller or the CLI can override them:
# workflow.yml
sample_size: 1000 # default 1000
reference: null # no value: omitted from the action
network: [asia, sports] # a list value, passed on as one value
steps:
- name: "Learn"
uses: "causaliq-discovery"
with:
action: "learn_structure"
sample_size: {{sample_size}}
reference: {{reference}}
cqflow run workflow.yml --param sample_size=5000 \
--param reference=networks/asia.graphml
A declaration binds as follows:
| Declaration | Value used when nothing supplies it |
|---|---|
sample_size: 1000 |
1000 |
reference: null |
none — the parameter is omitted from the action, so the action's own default applies |
network: [asia, sports] |
the list [asia, sports] |
A list may be written inline ([asia, sports]) or as a block list:
network:
- asia
- sports
The two spellings are equivalent. A list item must be text, a number or a
boolean, so a nested list and a null item are rejected. A list default
is one value: it never adds a matrix dimension, and {{network}}
delivers the list itself rather than one element per job.
Defaults are convenient when several steps need the same value, and they let a workflow be run standalone without supplying anything.
Four rules keep declarations honest:
- A declared parameter must be used as a
{{placeholder}}in at least one step'swithblock orname. An unused declaration is a validation error. - A
nullparameter may only be the whole value of a step parameter."data/{{reference}}.csv"would substituteNoneand is rejected. - A list parameter may also only be the whole value of a step
parameter. Interpolating a list has no text form, so
"data/{{network}}.csv"and a list element such as["{{network}}"]are rejected rather than substituted as text. - A placeholder that is not declared is a required parameter: a caller or the CLI must supply it, otherwise validation reports the name as an unknown template variable.
A declaration never adds a matrix dimension, and no parameter value is
part of a cache key, so a changed default does not invalidate existing
entries. Use matrix when each value must be a separate job.
matrix and steps are reserved. A name may not be declared as both a
parameter and a matrix variable. See
Reusable Workflows for parameters
passed to a called workflow.
Parameters and Defaults
Omitting an action parameter means that its default applies. Omitting it is also how a parameter that does not apply to the action is expressed — for example, a score type for a constraint-based structure learning algorithm. The two cases are deliberately indistinguishable, so a workflow never needs a null or "none" value:
with:
action: "learn_graph"
input: "data/asia.csv"
algorithm: "pc"
# randomise, sample_size and variable_types omitted: their defaults apply
Every default is a value the action could have been given explicitly. The
learn_graph action, for instance, accepts randomise: no-randomise,
sample_size: all and variable_types: impute as explicit spellings of
its defaults. That makes a default expressible in a matrix, comparable in a
filter, and passable to a called workflow:
matrix:
randomise: [no-randomise, var_best]
A literal null in a with block is rejected with a message asking you to
omit the parameter instead. A {{placeholder}} that resolves to null at run
time is still allowed: that is how a matrix wildcard marks a dimension as
not applicable (see
Null Values and Dimension Matching),
and how a workflow parameter declared with a null default reaches an
action as an omitted parameter.
The metadata recorded for a step follows the same rule: parameters that were supplied or defaulted are listed, parameters that did not apply are absent, and no recorded value is null.
Steps
Each step performs a single unit of work. Steps execute sequentially.
steps:
- name: "Evaluate Graphs"
uses: "causaliq-analysis"
with:
action: "evaluate_graph"
input: "results/graphs.db"
reference: "networks/asia/true.graphml"
Step Properties
| Property | Required | Description |
|---|---|---|
name |
Yes | Human-readable step name, unique within the workflow |
uses |
Yes | Action provider, or a relative path to another workflow |
with |
Yes | Action parameters, or a called workflow's parameters |
The uses property specifies which CausalIQ package provides the action
(e.g., causaliq-analysis, causaliq-discovery, causaliq-knowledge).
A value that starts with ./ or ../ calls another workflow instead of
an action; see Reusable Workflows.
A name may contain {{placeholder}} tokens. They are expanded for each
job, so a name can identify the matrix combination a step ran for; see
Step Names.
Actions
Actions are the reusable operations provided by CausalIQ packages. Each action has specific parameters documented in its package.
The action parameter within the with: block specifies which action to run:
with:
action: "merge_graphs" # Required: which action to perform
input: "results/graphs.db"
output: "results/merged.db"
Actions follow one of two patterns — create or aggregate — determined by whether the step reads a workflow cache. Steps that write plain files run once. See Action Patterns.
Matrix Expansion
The matrix feature runs steps across multiple parameter combinations, essential for comparative experiments.
matrix:
network: [asia, cancer, alarm]
sample_size: [100, 1000]
steps:
- name: "Learn Structure"
uses: "causaliq-discovery"
with:
action: "learn_structure"
network: {{network}}
sample_size: {{sample_size}}
output: "results/graphs.db"
This creates 6 jobs (3 networks × 2 sample sizes), each with a unique
combination of network and sample_size.
Matrix Behaviour
- Each combination produces a separate execution
- Matrix values are available as template variables
- Results are stored with matrix values as cache keys
Null Values and Dimension Matching
A null value on either side (target or entry) means the dimension is
not applicable and is treated as a wildcard — it always matches.
A missing key on the entry side (dimension absent from the input
cache) is also treated as a wildcard, so that caches with fewer
dimensions can be consumed by broader matrices.
matrix:
network: [asia, cancer]
llm_model: [null] # wildcard — match any llm_model
sample_size: [1K, 10K]
| Scenario | Target | Entry | Matches? |
|---|---|---|---|
| Target wildcard | null |
any value | Yes |
| Entry wildcard | "claude" |
null |
Yes |
| Both null | null |
null |
Yes |
| Missing key in entry | "1K" |
(absent) | Yes |
| Concrete match | "asia" |
"asia" |
Yes |
| Concrete mismatch | "asia" |
"alarm" |
No |
!!! tip "Separate caches for separate sources"
When aggregating entries from different sources (e.g. FGES and LLM
PDGs), store them in separate caches rather than one shared
cache with null dimensions. Use the list syntax for input: to
read from multiple caches:
```yaml
input:
- results/fges-pdgs.db
- results/llm-pdgs.db
```
This avoids entries from one source unintentionally matching
targets intended for the other source via null wildcard matching.
Template Variables
Template variables use {{variable}} syntax to reference workflow
parameters and matrix values within step parameters.
matrix:
network: [asia, cancer]
steps:
- name: "Learn Structure"
uses: "causaliq-discovery"
with:
action: "learn_structure"
network: {{network}}
dataset: data/{{network}}_10k.csv
output: results/graphs.db
Types
A value that is exactly one {{name}} token keeps the variable's native
type: a matrix number stays a number, and a list parameter stays a list.
Text with extra characters is substituted as text.
with:
randomise: {{randomise}} # a list parameter, passed through
dataset: data/{{network}}.csv # text: "data/asia.csv"
An action parameter declared as int, float or bool also accepts its
text spelling. The value is converted just before the action runs, so
seed: "42" and seed: 42 are equivalent. A str parameter keeps the
text you wrote.
Quoting
Ordinary values need no quotes. A whole-value template token is written without quotes:
sample_size: {{sample_size}} # valid: a template token
Quote a value only when YAML would otherwise read it as syntax or as a
different type. For example, version: "1.10" keeps the trailing zero,
and a value containing : must be quoted. See
Values and Quoting for the complete rules.
Variable Resolution Order
Variables are resolved in this order, with later entries winning:
- Workflow parameters: declared defaults
- Matrix variables: values from the current matrix combination
- Supplied parameters: values from the caller's
withor--param
Step Names
A step's name may contain {{placeholder}} tokens. They are expanded
for each job before the step runs:
matrix:
series: [asia, cancer]
steps:
- name: "Learn graph using bnlearn for {{series}}"
uses: "causaliq-discovery"
with:
action: "learn_bn"
series: {{series}}
output: "results/graphs.db"
Each matrix combination logs and records its own name, so
cqflow run workflow.yml --mode=run --log-depth=2 prints
Learn graph using bnlearn for asia and
Learn graph using bnlearn for cancer. The expanded name is also what
the completion summary and a called workflow's step list report.
A name is always text, so every value is expanded with its text form: a
number renders as 1000, a boolean as True, and a list as
['asia', 'sports']. A null value renders as None, so give a name
placeholder a scalar value and avoid a null-default parameter or a null
matrix wildcard in a name.
Naming rules:
- A placeholder in a name counts as a use of a declared parameter, so a
caller or
--parammay supply it. - Validation runs before the matrix is expanded, so a validation message shows the name as written.
- Names must stay unique after expansion. Two steps that expand to the same name in one job are rejected, because results are recorded by name.
Validation
Template variables are validated at parse time. Unknown variables cause a validation error, which is also how a missing required parameter is reported:
Unknown template variables: ['seed']. Available variables: ['dataset',
'network']. Declare a default value for each missing name as a top-level
workflow key, or supply it with 'with:' in the calling workflow or
--param.
A workflow that is called from another workflow also sees the caller's
parameter names, and the CLI passes --param names to the parser, so a
template may reference a value that is supplied at run time.
AGGREGATE steps also see the matrix keys derived from their input caches, so
{{network}} resolves from a learned graph cache even when the workflow
declares no matrix. When an input cache does not exist yet — it may be
produced by an earlier step — that step's variables are resolved at runtime
instead.