V I S O R

Quant Methods · intermediate · 8 min

The Random Control

This is the most important idea in Visor, and arguably the most important idea in honest strategy testing. Everything else — the gates, the verdicts, the whole "test it honestly" philosophy — rests on it. If you take one concept from this track, take this one: beating zero proves nothing; beating a scrambled version of yourself is the only test that means anything.

A Backtest Results panel with its robustness verdict, the check that separates a real edge from a lucky one

The wrong question and the right one

The tempting question about any strategy is "did it make money?" It is the wrong question. In a market that spent the period going up, almost anything that was long made money. Being long a rising market is not skill — it is participation. The return tells you the market went up and you were in it. It does not tell you your decisions — your entries, your timing, your signal — did any work at all.

The right question is "did this strategy's timing beat luck?" That is a comparison, and to make it you need something to compare against. The random control builds that something.

The null distribution

Here is the machinery. The engine takes your exact strategy and re-runs it 500 times, with its entry timing scrambled. Each scrambled twin keeps everything that isn't skill:

The only thing destroyed is the alignment between your signal and the price — when it chose to enter. Since when-to-enter is the strategy's entire claim to skill, each scrambled version is a copy of your strategy with the skill surgically removed and nothing else changed.

Those 500 skill-less returns form a null distribution — a picture of what returns this strategy earns by luck alone, from being in this market this many times with this exit. Your real result is then dropped onto that picture. The question the gate answers is simply: where does the real return land among its 500 randomly-timed twins?

To pass, the real return has to beat 95% of them. If a coin-flip on entry timing does about as well as your real signal, then your timing was not doing measurable work — the return came from being in the market, not from being smart about when.

The number that makes it concrete

From Visor's own tests: a strategy returns +94.8%. Read alone, that is a triumph — nearly a double. Run against its random control, its randomly-timed twins averaged +129.6%. The scrambled, brainless versions beat the real one. The +94.8% was never evidence of a good strategy; it was evidence that the market went up a lot, and the signal was, if anything, slightly worse than throwing darts. No amount of staring at the equity curve would have revealed that. The random control reveals it in one comparison.

This is the trap the gate is built to catch — the bull-market rider, a strategy that looks brilliant because it was long during a rally and makes its money from the rally, not the rule. The golden cross meets exactly this fate in The Golden Cross: a large headline return, beaten by a large fraction of its own scrambled versions.

Why it is not the only gate

The random control is the heart of the system, but it does not catch everything. A strategy that rides a slow drift present in the whole sample can sail through it — because the drift is in the scrambled twins too. That failure has a different fingerprint, caught by the out-of-sample split, which holds back the last 30% of history and checks the edge survives on data it was never measured on. And a strategy that was cherry-picked from many attempts needs the multiple-testing correction on top. The gates are a panel, not a single judge, and a strategy is only as credible as its weakest one.

The philosophy in one line

Most places that teach a strategy show you the version where it works. Visor shows you the strategy and then asks whether its timing beats a scrambled copy of itself — and frequently it does not. That is not the tool being pessimistic. It is the difference between a signal that carries information and a number that looks impressive because the market went up. Beating zero is trivial. Beating yourself-with-the-brains-removed is the whole game.

What to read next