What Is Overfitting? Strategies That Fit the Past Too Well in Backtests

What is overfitting?

Overfitting is when a strategy learns the coincidences and noise in past data instead of a real pattern. It looks great in the historical test and then fails to deliver live. It is like memorising the answers to an exam instead of understanding the subject.

How does it happen?

• Trying too many settings: If you try hundreds of window lengths, thresholds and filters and pick the best-looking one, that result is often luck.
• Adding too many rules: Adding a new rule to block every bad trade produces a model that only fits the past.
• Too little data: A result from a small number of trades is statistically weak.

Warning signs

1. Very high returns or a very smooth curve. Real strategies have bad periods too. See What is the Sharpe ratio?
2. Extreme sensitivity to settings. If a small change in the window length wrecks the result, the strategy is not robust.
3. Collapse on out-of-sample data. Good in the development period and bad afterwards is the classic sign.

How to avoid it

1. Split the data: Divide it into development and test periods. Choose settings only on the development period and look at the test period once.
2. Keep it simple: Rules with few parameters tend to be more robust.
3. Add costs: Commissions and slippage weed out weak strategies. See Commissions and slippage in backtests.
4. Validate in a virtual portfolio: Test forward with real prices. See the Backtesting and paper trading guide.
5. Watch for survivorship bias: See Survivorship bias explained.

Remember

A backtest is a screen, not proof. The risk of overfitting depends less on how good the result is than on how many attempts it took to find it.

This article is educational and not investment advice. Past results do not guarantee future results; trading carries a risk of loss.