Risk warning. Trading foreign exchange carries a high level of risk and is not suitable for everyone. This guide is educational and is not financial, investment, or trading advice. Never trade money you cannot afford to lose.
Backtests mislead through dirty data, overfitting, and missing costs. The defense is a fixed sequence: verify the data against a second source, validate out-of-sample on a period the strategy never saw, model spread and slippage, then run at least two months of demo trading before any real dollar.
Anyone with a laptop can produce a backtest that goes up and to the right. That is the problem. A backtest is a claim about the past, built by someone who already knows how the past turned out. Unless the test is designed to resist self-deception, it will confirm whatever the tester hoped to find.
There are three main ways a backtest lies. They are worth knowing individually, because each one requires a different check.
Lie number one: dirty data
Every backtest is only as good as the price series underneath it. Free and convenience price feeds are the usual suspects: candles can be missing, duplicated, or simply wrong, and the errors are quiet enough that most testers never look.
A concrete example of what "wrong" looks like: an entire dataset where the open of every bar equals the close of the same bar. That single defect flattens intrabar movement, changes where stops and limits would have filled, and can turn a losing strategy into an apparent winner. A strategy tested on that data is being graded on a market that never existed.
The defense is boring and non-negotiable: verify data quality before trusting any result built on it. Practical checks:
- Compare the same instrument and period across two independent sources. Meaningful disagreement means at least one source is wrong.
- Scan for structural defects: bars where open equals close across the whole file, zero-range candles, gaps that do not match weekends or holidays.
- Prefer data from a source close to execution, such as a broker or a dedicated data provider, over generic free feeds.
- Re-run the exact same backtest on the second source. If the result changes materially, the edge lives in the data errors, not in the strategy.
Lie number two: overfitting
Suppose you test a moving-average strategy and try every combination of two periods from 5 to 60, plus five stop sizes and five targets. That is thousands of variants. Even if none of them has any real edge, a few will show excellent results by pure luck, the same way a few people in a large coin-flipping contest will flip ten heads in a row.
Picking the best combination and calling it "the strategy" is called overfitting, or curve fitting. The rules have been molded to the accidents of one specific slice of history. The more parameters you tune and the more combinations you try, the more the final result describes the past and the less it generalizes to anything that comes next.
The defense is out-of-sample validation. Split the history. Optimize only on the first part, the training period. Then run the chosen settings, untouched, on the later part the strategy never saw. If performance collapses out-of-sample, the in-sample result was luck. A real edge does not need to be exactly as good out-of-sample, but it should survive. Two related habits help:
- Count your attempts. If 500 configurations were tested, the single best one is almost certainly inflated. Judge the family of results, not the champion.
- Prefer few parameters. A rule with 2 knobs that works acceptably is more trustworthy than a rule with 9 knobs that works beautifully.
Lie number three: the missing costs
Many backtests assume free, instant, perfect execution. Real trading charges three tolls on every position, and a thin edge can disappear entirely under them.
| Cost | What it is | Why backtests miss it |
|---|---|---|
| Spread | The gap between buy and sell price, paid on every single trade | Many testers use mid or bid prices only, so entries look better than any real fill |
| Slippage | The difference between the intended price and the actual fill, worst around news and fast markets | Historical data cannot show what liquidity was available at your moment of execution |
| Swap | The overnight financing charge or credit on positions held past rollover | Backtests often ignore it, yet it compounds on every multi-day trade |
The scale matters. A strategy that averages 3 pips of profit per trade is not a 3 pip strategy if the spread is 1.5 pips and average slippage is 0.5. Half the edge was fiction. Always re-run the test with realistic spread, a slippage allowance, and swap on held positions. If the strategy only works with zero costs, it does not work.
The honest sequence
Put the three defenses in order and you get a testing pipeline that is hard to fool:
- Clean data first. Verify the price series against a second source and scan for structural defects before running a single test.
- Out-of-sample test. Optimize on one period, validate on a later period the strategy never saw, with full costs modeled in both.
- Demo before dollars. Run the strategy on a demo or paper account for at least two months. This catches everything the backtest cannot: execution behavior, platform quirks, and your own discipline under real-time uncertainty.
Only after all three stages does a real dollar enter the conversation, and even then at small size with fixed risk per trade. The sizing rules for that step are covered in forex risk management basics.
This sequence is also your best defense as a buyer. Sellers of expert advisors lean on exactly the three lies above: pretty curves on unverified data, optimized parameters, and zero-cost assumptions. The checklist in how to evaluate a forex robot before you pay shows how to demand better evidence before spending anything.
Related guides
- How to evaluate a forex robot (EA) before you pay
- Forex risk management basics: position sizing before predictions
- Our trading resources and prompt packs
- Browse all guides · Our services
Disclaimer. Trading foreign exchange carries a high level of risk and is not suitable for everyone. This guide is educational and is not financial, investment, or trading advice. Never trade money you cannot afford to lose.