A backtest that cheats off tomorrow won't fail code review. So I fired a spike into the future

The scariest bug in a backtest (the thing that estimates a strategy's returns on past price moves) isn't a crash. It's accidentally using tomorrow's price. It's called lookahead bias, and if you let even a sliver of tomorrow's close leak into today's math, your strategy turns into a prophet. The backtest numbers glow. Then you ship it live and it melts on contact. And the code never throws a single exception. If anything the numbers get better, so you're in a great mood. (The worst kind of bug is the one that makes you feel good.) This time I was writing the backtest engine for a pairs-trading strategy, and before I even started coding, the "no cheating" design got into a fight at the planning stage. More precisely, plans that past-me had written were fighting each other. Shift once, or shift twice? The docs disagreed The "no cheating" implementation basically comes down to "slide a series over by one" (shift(1) in pandas). Today's decision shows up in tomorrow's trade - you reproduce that real-world lag in code. And my planning docs said this: - Overall plan (past-me, a few days ago): "Shift the hedge ratio beta by shift(1) . Shift the position too, so it fills the next day." - Latest handoff note (also me): "Wait, isn't that a double delay?" Right. Shift once for beta, shift again for the position, and you're now two days late in total. Here's today's whole point: Too few shifts (= cheating) and too many shifts (= trading a day late) both quietly move the numbers, and neither one raises an error. Cheating inflates your returns, over-delay deflates them. What comes out either way is a plausible-looking Sharpe ratio (the score for how good the returns are). You glance at the output, go "yeah, seems reasonable," and it sails through. Be honest - are you confident you'd catch one extra shift(1) in code review? I'm not. So I gave up on eyeballing it. I stopped deciding "where to shift" case by case, and pinned the invariant to one sentence Here's the approach I took. First, pin the rule you have to protect down to a single sentence. The realized PnL of bar t depends only on information up to the close of day t−1. And put the actual shift in exactly one place. The signal math (beta, spread, z-score, the buy/sell decision) can use everything up to today's close. But in the PnL calculation, exactly one shift(1) - "today's decision takes effect starting tomorrow." Collect the shift into one spot and the thought "wait, do I need a shift here too?" disappears entirely. A double delay can't happen either - there's only one place that shifts. But writing the policy in a docstring (the explanatory text inside the code) is just a promise. Promises get broken. By me, three months from now. Fire a spike into the future, and prove the past doesn't budge So I "proved" this invariant with a test. I'm fond of how it turned out, so let me show you. Test 1: future spike injection. Take the same price series twice, and into one of them only, inject a giant spike on some future day (day t+k) - like price +500%. Compute beta and z-score for each, and verify that every value before the spike is byte-for-byte identical (not approximately equal - numpy's array_equal , i.e. exact match). Whatever happens in the future, the past computation doesn't move by a single bit. If the beta window were accidentally including the future, the spike would "bleed" back into the past and break the match. I locked the other direction too: I also assert that beta on the spike day does change. Pin the rule "the compute window includes today's close" with a test, and if someone later adds a shift out of kindness, that test fails too. (An extra shift tends to be born of kindness.) Test 2: entries realize the day after the decision. Build synthetic data where the z-score crosses the entry threshold on day 5, and strictly verify that PnL is zero on day 5 and only starts on day 6. If "decide today, profit from today's move" ever happens, it's an instant fail. Now both "no cheating" and "not a day too late" are locked in as regression tests. From here on, if I refactor the engine and these tests stay green, I can at least say there's no time-direction cheating. Aside: the correct engine failed a test While I'm confessing - a sanity test that said "a mean-reverting series should turn a profit" failed at first. Bad luck on the RNG seed: with only 10 trades on the synthetic data, it just happened to land on the loss side. I ran the seed and parameters through a few combinations, swapped in synthetic data that reliably turns a profit, and that fixed it. I burned 30 minutes suspecting an engine bug - but what I should have suspected was the RNG in the test data. Takeaways Kill time-direction bugs with tests, not eyeball review. - Pin the rule you protect to one sentence - "bar t's PnL depends only on info up to t−1" - and collect the shift into one place - Fire a spike into the future and prove the past is byte-for-byte identical (catches a missing shift) - Also pin that today's value does change (catches an extra shift) - Strictly verify with synthetic data that entries realize the next day A cheating backtest will praise you. When it does, try firing one spike into the future. Top comments (0)