Paper versus backtest drift checker
This tool compares a backtest with a paper-trading run of the same length and shows how far they have drifted apart, in the number of trades and in the average result per trade. The gap between the two is the most useful measure of how optimistic the simulation was, and it tells you where to look next.
Try it
Fewer trades than the backtest implied. The entry conditions may be rarer live, or a signal that fires on a candle close is arriving later than the backtest assumed.
Average result per trade is well below the backtest. Fills may be worse than the cost estimate, or spreads wider than assumed.
Small sample. Fewer than 20 paper trades is too few to read much into. Keep going before drawing a conclusion.
The 0.7 and 1.3 bands are an illustrative rule of thumb for this tool, not an AKQX threshold. Use the same period length for both runs, and remember that a small number of paper trades is a noisy comparison.
How it works
Enter how many trades the backtest produced over a period, and the average result per trade after costs. Then enter the same two figures from a paper-trading run covering a period of the same length. The tool divides the paper figures by the backtest figures to give two ratios: paper trades as a share of backtest trades, and paper average as a share of backtest average.
It also multiplies each count by its average to show the total for the period in both cases, so you can see the combined effect of the two differences.
The messages underneath use two bands, below 70% and above 130%, to flag a large difference. Those bands are an illustrative rule of thumb for this tool. They are not an AKQX threshold, and a different person could reasonably choose different ones.
How to read the result
Read the two ratios separately, because they point at different problems. A low trade-count ratio says the entry conditions are rarer live than history implied, or that a signal based on a candle close is arriving later than the backtest assumed. A low average ratio says fills are worse than the cost estimate, or that spreads were wider than assumed.
A ratio near 100% is encouraging but not conclusive, especially from a short run. A ratio well above 100% deserves suspicion as much as comfort, since a few lucky trades can flatter a small sample.
Look at the small-sample warning. With fewer than twenty paper trades the comparison is noisy, and the honest response to a surprising number is to keep running, not to change the rules halfway through.
The example loaded above
The backtest produced 12 trades in a month at an average of 0.30 per trade, a total of 3.60. Paper trading the same rules for a month produced 7 trades at an average of 0.18, a total of 1.26. Paper trades are 58% of the backtest count and the paper average is 60% of the backtest average, both below the 70% band. So the live run found fewer opportunities and made less on each, which together turned 3.60 into 1.26. Those two flags are the starting points for investigating. The numbers are made up, to show the method.
What this tool does not do
- It cannot say why the two runs differ. It points at where to look.
- It compares aggregates and ignores the order of trades, drawdowns and which trades differed.
- It assumes the periods are comparable. If the backtest period was calm and the paper run was volatile, the comparison is partly about the market, not the strategy.
- A small number of trades makes any ratio unreliable, whatever the tool prints.
Educational only. AKQX is software only. It is not a SEBI-registered Investment Adviser or Research Analyst and gives no investment advice. This calculator does arithmetic on the numbers you enter. It does not tell you what to trade, how much to risk or whether any setup is sound. See the risk disclosure.
Questions about paper vs backtest drift checker
What is drift between a backtest and paper trading?
Drift is the gap between what a backtest predicted and what the same rules did on live prices with virtual money. It measures how optimistic the simulation was, and it is usually more informative than either result alone.
How long should I paper trade before comparing?
Long enough to see a meaningful number of trades and more than one kind of market day, rather than a fixed number of days. A strategy that trades twice a week needs much longer than one that trades twenty times a day.
Is some drift normal?
Yes. Live prices bring spreads, delays and rejections that history does not contain, so paper results are usually somewhat worse. What matters is whether the drift is small and explicable.
Should I change my rules if the drift is large?
Understand it first. Find out whether trade count or average result is the problem, and why. Changing rules in the middle of a run means you are no longer comparing like with like.
Related
Take the number into a real test
A free account includes 10 backtests a month and 1 paper algo, so you can check a setting against history and live prices.