Why the workspace counts your trials against you

Run one strategy, once, and it reports a Sharpe ratio of 1.4. Run sixty variations of the same idea — different lookback windows, a handful of alternative universes, a few different entry thresholds — keep the best one, and it also reports a Sharpe ratio of 1.4. Both numbers look identical on a slide. They are not the same finding, and a workspace that reports them the same way is quietly misleading whoever reads the second one.

The reason is ordinary sampling variation. Test enough variations of anything against noisy market data and one of them will look good by chance, whether or not any of them carries a real edge. The best of sixty genuinely random strategies, tested against a market with no exploitable pattern at all, will very likely show a Sharpe ratio well above zero — not because it found anything, but because the maximum of sixty draws from noise tends to drift upward as the number of draws grows. A reported Sharpe ratio that does not say how many configurations were searched to find it is missing the single most important piece of context for judging whether the number means anything.

What the deflated Sharpe ratio actually does

The correction for this is the deflated Sharpe ratio, developed by Bailey and Lopez de Prado (2014). It takes the observed Sharpe ratio and asks a sharper question: given how many configurations were actually tried, and given how skewed and fat-tailed the strategy’s returns are, what is the probability that its true Sharpe ratio exceeds zero at all? A Sharpe ratio of 1.4 chosen from one honest attempt survives that question comfortably. The same 1.4, chosen as the best of sixty attempts, survives it far less well — sometimes not at all.

This is not a cosmetic adjustment applied after the fact. It changes which strategy a researcher should actually trust, and it can turn what looked like the strongest candidate in a table into one of the weaker ones once the number of attempts behind each entry is accounted for. That is also why experiment comparison tables in the workspace are sorted by deflated Sharpe ratio, not raw Sharpe ratio — sorting by the raw number is exactly how the most over-searched configuration wins, every time, regardless of whether it is actually the best strategy.

The trial ledger is what makes the correction honest

A deflation calculation is only as trustworthy as the trial count feeding it. If a researcher can quietly under-report how many configurations were tried — leaving out the ones that did not work, forgetting the securities considered and dropped — the correction understates itself and the reported Sharpe ratio looks stronger than it should. This is the entire reason the trial ledger exists as its own append-only record, separate from the experiment results themselves: every configuration attempted is logged as it happens, including the ones that were tried and discarded, and there is no operation in the system to edit or delete an entry once it has been written. Closing a ledger blocks any further appends outright.

The ledger does not, and cannot, catch a researcher who explores ideas entirely outside the tracked workflow — in a side notebook, or by hand, before bringing a final idea into the workspace. No system can correct for a search it never observed. What the ledger guarantees is narrower and more useful: for everything that was run inside the workspace, the trial count behind any reported deflated Sharpe ratio is complete and cannot have been edited after the fact to look more favourable.

Why this belongs in the reporting, not just the methodology

It would be possible to compute the deflated Sharpe ratio quietly, use it internally to decide which strategy to pursue, and then report only the raw Sharpe ratio in a final document — the number a reader is more likely to find intuitive and impressive. The workspace does the opposite deliberately. The robustness panel renders the raw Sharpe ratio directly above the deflated Sharpe ratio, so the gap between the two is visible rather than hidden, and a top-level verdict of supported, not supported, or insufficient evidence always carries the reminder that none of this is investment advice.

Showing both numbers side by side, rather than only the more flattering one, is a small design choice with a specific purpose: a reader should be able to see for themselves how much of a claimed edge survives once the search behind it is accounted for, instead of having to trust that the correction was applied correctly somewhere upstream. An evidence-based research tool that hides the size of its own multiple-testing problem is not meaningfully different from one that never checked for it at all.

What this looks like in practice

A researcher exploring a momentum signal across a universe of fifty stocks, three lookback windows and four position-sizing rules has run up to six hundred implicit combinations before they have consciously “tried” anything — every combination of parameters is a configuration, whether or not it was run to completion. The trial ledger tracks the ones that were actually executed, and the deflated Sharpe ratio computed at the end reflects that count, not the researcher’s intuitive sense of how much searching felt like it happened.

This tends to be uncomfortable the first time a researcher sees it. A strategy that looked like a clear winner under its raw Sharpe ratio can report a deflated Sharpe ratio close to zero, or a probability of backtest overfitting high enough to flag the whole exercise as inconclusive. That discomfort is the corrected number doing its job: it is telling the researcher something the raw number was never capable of telling them, which is exactly why the workspace treats the corrected figure, and the search that produced it, as the more important part of the result to report.

The point is not that extensive searching is illegitimate — trying many configurations is often how a genuine edge gets found. The point is that the search itself is part of the evidence, and a result that hides how much of it happened is a weaker result than it appears, no matter how good the headline number looks on its own.