Building a research workflow you can re-run

A researcher builds a pipeline for a signal-research project: load a universe, run a dataset integrity check, engineer momentum features, train a classifier, walk-forward validate it, backtest the resulting strategy. It works. Three months later, a teammate asks to run the same pipeline on a different universe to see whether the signal generalises. The researcher opens their notebook, finds six cells that were rerun out of order at some point, two parameters that were changed and never changed back, and no record of which combination actually produced the result the teammate is trying to reproduce. Handing over “the pipeline” means handing over an afternoon of archaeology.

This is the problem a workflowWorkflow An ordered, validated, versioned list of steps, each naming a method and its parameters, that can be run once or repeatedly on new data. is built to solve: not running a set of steps once, but describing them in a form specific enough that someone else — or the same researcher, six months later — can press run and get the same thing back.

A specification is not a run

The distinction that matters most here is between the specification of a workflow and one particular execution of it. The specification is an ordered, validated list of steps: each step names a method from the method catalogue and binds its parameters. That is all a workflow specification is — a plan, not a result. An execution is what happens when that plan is actually run against a specific project’s data at a specific time, producing a specific set of outcomes.

Conflating the two is the exact mistake in the notebook example above: the researcher had a pipeline that had run successfully once, which is not the same thing as having a specification someone else could run again. A workflow specification is reusable precisely because it does not carry any of the state from a particular run — no cached intermediate results, no assumption about what happened to be loaded in memory. It is a description of intent, checked for internal consistency before it is ever executed.

Validation happens before running, not during

A workflow is checked for problems at the moment it is saved, not discovered mid-run when a step three-quarters of the way through fails because an earlier step’s output was never produced. Problems separate into two categories.

Blocking problems stop the workflow from being saved at all: no name, an unknown state, zero steps, duplicate step ids, a step that names a method id the catalogue does not recognise, an invalid parameter binding, a depends_on reference to a step that does not exist, or a dependency cycle. None of these can be fixed once the pipeline is already running — they have to be fixed in the specification, before anyone presses run.

Warnings flag problems that do not block saving but are worth a researcher’s attention: a step whose declared inputs nothing earlier in the workflow actually produces, or a step that references a catalogued method not yet implemented in this deployment. A workflow can be saved with warnings. It cannot be saved with blocking errors.

This upfront validation is what separates a workflow from a script that happens to work today. A script can have a broken dependency and simply fail loudly when it reaches that line — annoying, but at least honest in the moment. A workflow catches the same broken dependency before anyone runs anything, at the point where it is cheapest to fix and where the person fixing it still remembers why they wired the steps together that way.

Versions accumulate, they do not overwrite

Every time a workflow specification is saved, the version number increases and the previous full record is appended to a history array. Nothing about an earlier version is deleted or overwritten. A researcher who tightens the embargo setting on a feature-engineering step, saves, then later widens it back for a different experiment, has three versions on record — the original, the tightened one, and the reverted one — not one specification silently mutating underneath a version label that no longer means anything specific.

This matters for the same reason a content hashContent hash A SHA-256 hash of a fixed set of an experiment's inputs, not its results or timestamp, used to detect whether a supposedly identical re-run produced a different answer. on an experiment record matters (see /quantlab/practice/reproducible-research for how that mechanism works): a result is only traceable if the exact configuration that produced it can be recovered later. A workflow whose specification changes in place, with the old version discarded, breaks that traceability the moment anyone edits a step. Accumulating versions instead of overwriting them means a researcher can always ask “which version of this workflow produced the result I am looking at” and get a real answer, not a guess.

One detail worth being precise about: moving a workflow between lifecycle states — draft, published, archived — does not itself bump the version number. That transition is a decision about the workflow’s status, not a change to what it does, and treating it as a new version would clutter the history with entries that carry no actual specification change.

The five outcomes of a step, and why “it failed” is not enough

When a workflow actually runs, each step in it resolves to exactly one of five outcomes:

Outcome	Meaning
ok	Ran and produced a result.
unavailable	Ran, could not produce a result, and states why.
skipped	Not attempted — disabled by the researcher, or unimplemented in this deployment.
blocked	Not attempted because a dependency it needed had already failed.
failed	Attempted and raised an error, with a traceback recorded (capped at six frames).

The distinctions between these matter more than they might first appear. unavailable and failed are both “did not produce a result,” but they mean very different things to a researcher reading the run afterward. unavailable means the method ran correctly and concluded, honestly, that it could not answer the question — for example, a VWAP scheduling calculation that refuses to run without a volume profile. failed means something broke that was not supposed to. Collapsing both into a single “error” status would erase a distinction a researcher genuinely needs when deciding whether to trust the rest of the run.

blocked versus skipped matters for the same reason. A step marked skipped was never going to run regardless of what happened elsewhere — it was disabled, or the method it names is not implemented in this deployment. A step marked blocked would have run, but a dependency upstream failed first. Knowing which one happened tells a researcher exactly where in the pipeline to look when something is missing from the output.

Each run record also captures which parameters were actually used — resolved defaults plus any overrides, not merely what the researcher originally typed — and what data the step had available to it at the time. This is the same principle behind the reproducibility checklist covered in /quantlab/practice/reproducible-research: a result is only as trustworthy as the record of exactly what produced it, and “what parameters did this step actually use” is one of the most common gaps in a hand-run analysis.

Starting from a template rather than a blank page

Five built-in templates exist precisely because most research projects start from one of a small number of recognisable shapes rather than a genuinely novel structure: Blank workflow, Signal research, Volatility modelling, Portfolio construction, and Pairs and relative value. Each template is a named, ordered method list with per-step notes explaining the quantitative rationale for that step’s presence — not just what it does, but why it belongs at that point in the sequence.

Starting from a template does not remove the need to validate the resulting specification once it has been adapted to a particular project’s universe and data. A template gives a sound starting structure; it does not exempt the researcher from checking that the adapted version is still internally consistent once steps have been added, removed or rewired.

What “reusable” actually means here

A workflow you can rerun is not one that happened to work once. It is a specification that has been checked for structural soundness before anyone runs it, that accumulates its history rather than overwriting it, and that reports exactly which of five distinct things happened to each step when it did run. That combination is what lets a colleague, a reviewer, or the same researcher a year later pick up the specification and trust that running it again means something — rather than repeating the notebook archaeology this article opened with.