Backtesting trading strategies without fooling yourself

A backtest replays a written rule over past market data and records what the rule would have done. It does not tell you what will happen next week. At best it tells you whether an idea had a measurable edge under past conditions, after realistic costs, on enough trades to be more than luck. That is still very valuable, because most intraday ideas fail that test and you find out without risking capital.

The key word is written. If a rule needs your judgment at the moment of entry, a backtest can only test a simplified version of it. Accept that and test the simplified version honestly rather than pretending the computer captured your intuition.

Here is the kind of rule that works well for a first test. It uses only information available at the time of entry.

• Universe: US stocks priced $2 to $20 with average daily volume above 500,000 shares.
• Filter at 9:30 ET: gap of at least 5% from the prior close and premarket relative volume above 3.
• Entry: buy the first break of the premarket high between 9:30 and 10:30 ET.
• Stop: a fixed distance below the entry or below VWAP, whichever is closer.
• Target: 2R, with a time exit at 11:30 ET if neither stop nor target is hit.

Every element is defined, and that is what makes the result meaningful. The backtesting software in DayTradingSoftware is built around exactly these building blocks: entry rules based on gap %, RVOL, a premarket high break or a VWAP cross, plus a stop, a target and a time exit, run on historical intraday data.

Overfitting happens when you keep adjusting parameters until the past looks great. Test enough combinations of gap %, RVOL threshold, stop distance and exit time and one of them will look excellent by chance. The more knobs you turn, the more likely the winning version is describing noise.

A practical defense is to limit yourself. Decide in advance which two or three parameters you will vary and over what range, and look for broad plateaus rather than sharp peaks. If a gap filter of 5% shows a positive result, 4% and 6% should look similar. If only 5.3% works, you have found a coincidence.

• Change one parameter at a time and write down every version you tried.
• Prefer round, logical thresholds over oddly precise ones.
• Distrust any result that collapses when a parameter moves slightly.

Split your history before you start. Build and adjust the rule on the earlier part, for example the first 70% of the period, and leave the last 30% untouched. When you are done, run the final version once on the untouched portion. If the edge disappears there, the honest conclusion is that the rule does not hold up. Running the out-of-sample test again and again after adjustments quietly turns it into more in-sample data.

Many intraday strategies trade small caps, and small caps are exactly where companies get delisted, reverse split or merge. If a dataset only contains stocks that still trade today, your test misses the names that collapsed, which are often the ones a gap strategy would have bought. The result looks cleaner than reality. When you evaluate any data source, ask whether delisted symbols are included and whether prices are adjusted for splits in a way that matches what you would have seen on the day.

Look-ahead bias means using information that was not available at the moment of the decision. Classic intraday examples include filtering on the full day's volume when you entered at 9:45 ET, using the day's high to define a breakout level, or ranking stocks by their closing change. A rule that selects stocks at 9:30 ET may only use data up to 9:30 ET. It sounds obvious, and it is still one of the most frequent errors in homemade spreadsheets.

Fast moving small caps rarely fill at the exact breakout price. A backtest that assumes perfect fills overstates results, sometimes completely. Here is a worked calculation with hypothetical numbers.

Suppose a rule averages a gross gain of 4 cents per share per trade on a $6 stock, trading 1,000 shares. That is $40 per trade before costs. Now add realistic friction: 2 cents of slippage on entry, 2 cents on exit and half a cent per share in commissions and fees each way. Total cost is 5 cents per share, or $50 per trade. The rule that looked profitable now loses about $10 per trade.

• Gross result: 1,000 shares x 4 cents = $40.
• Slippage: 1,000 shares x (2 cents + 2 cents) = $40.
• Commissions and fees: 1,000 shares x (half a cent x 2) = $10.
• Net result: $40 minus $50 = minus $10 per trade.

Always run a test at least twice, once with your best estimate of costs and once with costs doubled. If the strategy only survives the optimistic version, treat it as unproven.

Twenty trades say almost nothing. With a win rate around 50%, random variation over 20 trades can easily swing the observed rate between roughly 30% and 70%. As a rough working rule, look for at least 100 trades before drawing conclusions about a simple setup, and more if the strategy depends on rare, large winners.

Also check how the trades are distributed. A test with 150 trades where 60 come from one week in a hot market is really testing that week. Break the results down by month and by market regime and see whether the edge exists in most periods or just one.

Win rate alone is misleading. Focus on figures that combine wins, losses and size.

• Expectancy in R per trade, after costs.
• Profit factor, gross wins divided by gross losses.
• Maximum drawdown in R, and how long it lasted.
• Average R of winners and losers.
• Longest losing streak, which tells you what you need to sit through emotionally.

For a worked calculation of expectancy: with a 40% win rate, an average winner of +2R and an average loser of minus 1R, expectancy is 0.40 x 2 minus 0.60 x 1, or +0.20R per trade. On a hypothetical $100 risk per trade that is $20 per trade before costs, which shows how quickly the slippage example above matters.

A backtest is a hypothesis. The next step is to trade the rule live at small size and record every trade in a trading journal with the same setup tag. After 30 to 50 live trades, compare the live expectancy with the backtest. If live results are much worse, the usual culprits are slippage, hesitation on entries and skipped trades. DayTradingSoftware stores the scan state at entry for each journal trade, so you can see whether your live trades actually met the conditions you tested.

Use a position size calculator so every trade risks the same amount, otherwise R based comparisons stop being meaningful. If you are still building your first setup, start with our guide on how to find premarket gappers and come back to testing once the rule is written.

• The rule uses only data available at entry time.
• Costs and slippage are included, and the test survives doubled costs.
• At least 100 trades, spread over different months.
• Results are stable when parameters move slightly.
• The final version passed one untouched out-of-sample period.
• Live trades at small size roughly match the test.