Look-ahead bias and data leakage: the ten ways it happens

A backtest with a Sharpe ratio of 2.4 is either a genuine discovery or a plumbing error, and the two look identical from the equity curve alone. Look-ahead biasLook-ahead bias Using information in a calculation that would not actually have been available at the time the decision it informs was made. is not one mistake; it is a family of ten distinct mechanisms by which information from after a decision point reaches the data used to make that decision, and each one has a specific fix. Treating “leakage” as a single vague risk to “watch out for” is how it survives a code review — naming the ten specific ways it happens is what actually catches it.

1. Using a shuffled or random train/test split

Financial data is sequential: today’s price depends on yesterday’s, and a model’s job is to say something useful about a future it has not seen. A random (non-chronological) split scatters rows from every point in the timeline into both the training and test sets, so a model trained on rows from January, March and October of a year is tested on rows from February, June and November of the same year — rows whose neighbours it has already memorised. Quant Compose’s pipeline defaults hold chronological_split: True; switching it off deliberately turns walk-forward evaluation into ordinary random-sample cross-validation, and the workspace’s own leakage warnings flag exactly this. The full mechanics of why this specific failure inflates every metric are covered in why random splits break on time series.

2. Fitting a scaler, encoder or imputer on the full dataset

A StandardScaler fit on the combined training-and-test data computes its mean and standard deviation using test-set values. The transformed training features then carry statistical information about the test set before the model has seen a single test row — a subtle leak because the scaler’s parameters, not the raw data, are the vector. Quant Compose’s scaler_train_only default fits the transform on the training split alone and refits per fold inside walk-forward, so no fold’s transform ever sees a later fold’s data.

3. A missing purge gap around the train/test boundary

A label built at row $t$ that depends on prices up to $t+h$ (a forward return, say) shares information with any test row inside that horizon. If the last training row sits at $t$ and the first test row sits at $t+1$, and the label horizon $h$ is 5 days, the training label at row $t$ was built using price information through $t+5$ — three days into the test period. Quant Compose’s purge rule, purge_gap = max(embargo, target_horizon), floors the gap at the label horizon specifically because the horizon itself, not just a user-configured embargo, determines how far forward a label’s information reaches. Skipping the purgePurge Removing training rows whose label window overlaps a test block, so a model never trains on information the test period's outcome depends on. step lets the training set effectively “know” the outcome of nearby test-period events, per Lopez de Prado (2018), ch. 7.

4. A missing embargo after the test block

Purging removes training rows whose label window overlaps the test block; embargoing goes further and removes a further stretch of rows immediately after the test period, because serial correlation in financial returns can still let information leak forward even without a directly overlapping label. Quant Compose’s embargoEmbargo Removing the rows immediately after a test block, where serial correlation would otherwise let information leak backward into training. parameter defaults to 5 (days), clamped between 0 and 60. An embargo of zero with a highly autocorrelated series is a residual leak that a purge gap alone does not close.

5. Point-in-time misalignment on fundamental or reported data

A quarterly earnings figure is reported for a fiscal period ending, say, March 31, but is not actually released to the market until early May. A dataset joined on fiscal-period-end date rather than release date hands a model a feature it could not have known for five weeks. This is why point-in-timePoint-in-time Stamping every record with the date it was actually knowable, so a later restatement or correction cannot leak into an earlier decision. alignment — stamping every record with the date it was actually knowable — is treated as a distinct step in Quant Compose’s data-acquisition methods rather than folded into ordinary date-joining logic.

6. Survivorship bias in universe construction

A universe built from today’s index constituent list and applied across ten years of history silently excludes every company that was removed from the index along the way — largely companies that performed badly enough to be delisted or dropped. A backtest run against that universe is systematically biased toward the survivors. Survivorship biasSurvivorship bias The distortion introduced when a universe only contains securities that existed to the present day, silently excluding the ones that failed or delisted. is technically a universe-construction failure rather than a per-row leak, but it produces the same symptom as the other nine: a number that looks better than it should because the sample was chosen with knowledge of the outcome. Quant Compose’s projects start with an empty universe that must be explicitly applied, and every stage’s results explicitly inherit whatever bias that choice carries.

7. Corporate-action leakage through an unadjusted price series

A stock split that has not been applied to historical prices produces an apparent overnight price collapse that has nothing to do with the market. If that raw series is used to construct a momentum or volatility feature without correcting for the split first, the feature encodes the mechanics of the corporate action rather than anything about the security’s actual behaviour — and because a split is a scheduled, well-known event, a model that stumbles onto this artefact by chance can look unreasonably predictive in-sample while explaining nothing that would generalise. The corporate-action check in Quant Compose’s catalogue is detection only, currently unverified against a published reference case, and flags the suspected jump rather than silently correcting it.

8. Reusing the test set across repeated model iterations

Every time a researcher looks at test-set performance, adjusts a hyperparameter, and re-evaluates on the same test set, the test set stops being an honest measure of generalisation and starts being a target the researcher is implicitly optimising against — a leak of information through the researcher rather than through the data pipeline. This is the mechanism behind the broader multiple-testing problem: the more configurations tried against the same held-out data, the more likely one appears to work by chance alone. Quant Compose’s Deflated Sharpe Ratio and Probability of Backtest Overfitting statistics exist specifically to correct a headline result for how many such trials were actually run, rather than reporting the best result as if it were the only one tried.

9. A cross-sectional feature computed from today’s universe applied to past dates

A cross-sectional rank, percentile or z-score — a security’s momentum relative to its peers on a given date — is only valid if computed against the universe that actually existed on that date. Computing it against today’s constituent list and applying the result to five years of history means every historical value is silently informed by which companies survived to be included in today’s list, compounding a universe-level survivorship problem into a feature-level one.

10. Overlapping labels inflating the effective sample size

A label with a horizon longer than one period — say, a 20-day forward return recomputed at every daily row — produces training examples whose labels overlap heavily with their neighbours’ labels. The rows are not independent observations even though they are counted as though they were, and a model trained on them, or a statistic computed across them, effectively has far fewer genuinely independent data points than its row count suggests. Quant Compose’s pipeline_warnings() explicitly flags a label horizon greater than 1 period for this reason, and label-uniqueness weighting — a method in the workspace’s catalogue that estimates how many independent observations a set of overlapping labels is actually worth — exists to correct for it directly rather than pretend the row count is the sample size.

Why the fix is procedural, not a checklist item

Each of the ten mechanisms above has a specific structural fix rather than a general one: chronological splitting for #1, train-only scaler fitting for #2, a purge gap sized to the label horizon for #3, an embargo for #4, release-date joins for #5, historically accurate constituent lists for #6 and #9, corporate-action detection for #7, out-of-sample trial accounting for #8, and label-uniqueness weighting for #10. Quant Compose’s pipeline defaults — chronological_split, scaler_train_only, no_shuffling, purged_embargo, and the fixed data-preparation order described in Preparing financial time series — are leakage-safe by default specifically because each default closes one of these ten mechanisms, not because a single global setting happens to catch all of them at once.

A worked comparison

Consider a classifier trained to predict a 5-day-forward directional move, with a naive random 80/20 split versus a chronologically split, purged and embargoed version, on the same underlying data. The random split reports a mean out-of-sample ROC AUC of 0.71 across folds. The purged, embargoed, chronological version reports 0.54 — barely above the 0.50 baseline of a coin flip. Both numbers came from the same features and the same model family; the 0.71 is not a better model, it is mechanisms #1, #3 and #4 operating simultaneously, and the 0.54 is the number that would actually have to survive contact with new, unseen data.