Summarise Columns and Output Formats¶
This note is the decision record for the summarise table model: how the
columns of a summary table are named, which input cache supplies a column,
how derived columns are computed, and how the presentation formats render
the table. The user guide
Summarising Results documents the same
contract for users.
Column model¶
A summary table is an ordered list of columns:
- Grouping dimensions first, in workflow
matrixorder. - Metric columns next, one per entry of the
metricparameter, in the declared order.
Each metric column is either a base column, computed from the metric values of one or all input caches, or a derived column, computed by a formula over the columns that precede it.
Aliases¶
A numeric index appears in exactly one place: the one-based source index
that names an input cache (f1.mean[2]). A column is never addressed by
its position. Instead a metric declares a short alias with as:
Aliases are optional, unique, must be identifiers and may not shadow a formula function. An alias names the column in later formulas and becomes its default label. Because formulas use names rather than positions, a numeric literal is always a number and reordering the metric list cannot change what a formula means.
Metric grammar¶
| Form | Meaning |
|---|---|
<field>.<stat> |
The statistic over every input cache (union) |
<field>.<stat>[<source>] |
The statistic over one input cache |
<field>.argmax.<dimension>[<source>] |
The winning dimension value of one input cache |
<field>[<source>] |
The single distinct value of a field |
<formula> |
A derived column |
Any of these forms may declare an alias with a trailing as <name>. An
entry that holds an operator or a function call is a formula; anything
else is a metric specification, so the two can never be confused.
Supported statistics are mean, sd, count, min, max, median
and unique. unique is action-only: it reports the single distinct
value of a field and rejects a field whose values disagree, which is what
a per-network node count needs. A base column without a source index
keeps the original union behaviour, so existing workflows are unchanged.
Joining input caches¶
The caches are joined on the grouping dimensions: a row is one matrix combination, and a source-qualified column takes its values from the entries of that cache which match the row's dimensions.
A metric that names its input cache therefore requires every entry of that cache to carry every grouping dimension. The workflow scan treats a missing dimension as a wildcard, which is right for a union but wrong for a join: an entry without the dimension would contribute to every row. Such an entry is rejected with an error naming the cache and the dimension.
Derived columns¶
A formula is a Python arithmetic expression over the aliases of the
columns that precede it, parsed with ast and evaluated against a
whitelist:
| Element | Supported |
|---|---|
| Operators | +, -, *, /, unary +/-, parentheses |
| Literals | numbers (0.5, 100) |
| References | aliases of earlier columns |
| Functions | abs, max, min, round |
metric:
- f1.mean[1] as bnsl
- f1.mean[2] as llm
- f1.mean[3] as fusion
- fusion - max(bnsl, llm) as gain
- round(gain * 100, 1) as gain_pct
Every element is unambiguous: a number is a number, a name is a column. References are backward only, so a cycle is impossible, and an unknown name fails validation with the alias it should have declared.
Missing values propagate as they do in a spreadsheet: arithmetic over an
empty cell yields an empty cell, min and max ignore empty values, and
a division by zero yields an empty cell.
Presentation formats¶
The output extension selects the renderer:
| Extension | Renderer | Applies format/footer |
|---|---|---|
.csv |
Raw table | No |
.tex |
LaTeX tabular fragment |
Yes |
.md, .markdown |
Markdown table | Yes |
CSV output always holds raw values because a plot step reads the file
with pandas.to_numeric; formatted cells or a footer row would deform the
chart. The presentation formats are fragments, not documents: .tex
emits the tabular environment for \input inside a paper's own table
float, and .md emits the bare table so it can be embedded anywhere.
headers labels every column of the table, so it holds one entry per
grouping dimension followed by one per metric; format and footer use
the same column order. Headers are free text and deliberately independent
of aliases: a header may be narrative or hold markup, it may differ from
the column's alias, and renaming a header cannot break a formula. Without
headers, a column's default label is its alias, or its specification text
when it declares none.
format entries are Python format specifications (.3f, +.3f, d),
validated when the parameters are parsed. Width padding is applied
literally but is rarely useful, because both renderers align the columns
themselves; the type character decides the alignment (s left, numeric
right, an explicit </^/> overrides it).
footer entries are blank, a statistic name (mean, sd, count,
min, max, median) or literal text. A statistic is computed over the
column's row values, so the footer of a derived column is the mean of the
derived values rather than the formula applied to the footer cells.
LaTeX output escapes markup (_, %, &, #, $, {, }, ~, ^,
<, >, \) and rules the table with \hline after the header and
before the footer. Markdown output escapes |, \, < and newlines, and
derives its alignment row from the format type; it has no rule, so the
footer is simply the last body row.
Module layout¶
The action keeps the model in small modules beside the original ones:
| Module | Responsibility |
|---|---|
columns |
Column specifications, aliases, label and source-index validation |
formula |
Derived-column parsing, validation and evaluation |
stats |
Base-column statistics and per-source value lookup |
present |
Display columns shared by the renderers, cell formatting |
footer |
Footer cells |
render_latex, render_markdown |
Emitters |
render |
Renderer selection, file writing |
collect, report |
Per-source value collection |
See Also¶
- Summarisation Paradigm — Aggregation model
- Summarising Results — User guide