A setup wins 8 of its first 10 backtested trades and it's tempting to call it proven. It isn't — not yet, and not because the setup is necessarily bad, but because 10 trades is nowhere near enough data for that win rate to mean what it appears to mean. Sample size is the single most under-respected variable in backtesting, and it's the reason so many setups that looked unbeatable in testing fall apart within their first month live.
Why small samples lie convincingly
Flip a genuinely fair coin 10 times and there's a real, non-trivial chance you'll get 7 or 8 heads purely by variance — nothing about the coin changed, the sample was just small enough for randomness to dominate the result. Trading setups behave the same way. A setup with a true 50% win rate can easily produce a 10-trade sample showing 70% or 80%, and a setup with a true 45% win rate — a losing setup, if the risk-reward isn't generous enough — can look identical to a winning one over the same small sample. Ten trades cannot distinguish a good setup from a lucky bad one. Neither, usually, can twenty.
So what sample size is actually enough?
There's no single magic number that applies to every setup, but the useful floor most serious traders converge on is around 30 to 50 trades before a win rate and expectancy start to stabilize into something you can act on with real confidence — and even that's a floor, not a finish line. A setup that only triggers a handful of times a month might need several months of backtesting to reach that floor; a setup that fires daily can reach it in weeks. The right benchmark isn't a calendar duration — it's trade count.
This is exactly why mysetuplap's Strategy Lab treats minSample as a real, explicit gate rather than a suggestion — a strategy hypothesis defaults to needing 20 trades minimum before it's even eligible to be judged SUPPORTED or REJECTED, and every strategy test records its actual sampleSize alongside its verdict. A setup with a strong-looking expectancy but a sample under that threshold gets marked INCONCLUSIVE, not SUPPORTED — because a promising result on too small a sample isn't evidence yet, it's a hypothesis still waiting on data.
Win rate stabilizes slower than you'd expect
The mathematically uncomfortable part of small-sample backtesting is how slowly the "true" win rate actually reveals itself. Even at 30 trades, a setup with a genuine 55% win rate can easily show anywhere from roughly 40% to 70% in that specific sample — the confidence interval around a win rate estimate shrinks as the square root of sample size, which means going from 10 trades to 40 trades (4x the data) only tightens your estimate by roughly half, not by four times. Doubling your confidence takes quadrupling your sample. This is exactly why a setup that "worked great" over 15 backtested trades deserves real skepticism, not celebration — the sample simply hasn't earned that conclusion yet.
Expectancy needs the sample even more than win rate does
Win rate alone is already unstable at small samples — expectancy, which multiplies win rate by average R-multiple, compounds that instability, because now a single outlier trade (one unusually large win or loss) can swing the entire average dramatically. A 25-trade backtest with one 8R outlier win can show a wildly inflated expectancy that has nothing to do with the setup's repeatable edge and everything to do with one exceptional trade that may not repeat. Larger samples dilute the influence any single trade has on the average — which is precisely the point.
The baseline comparison that catches false edges
Sample size alone isn't the whole story, either — a setup can clear 50 trades and still not prove anything if there's nothing to compare it against. This is why mysetuplap's Strategy Lab tests every hypothesis against a baselineSample and baselineExpectancyR pulled from your broader trading, not in isolation — a setup showing +0.6R expectancy sounds good until you learn your baseline across everything else you trade is +0.5R, at which point the "edge" you thought you found is barely distinguishable from your average trade. A real edge has to beat your baseline, not just be positive in isolation.
What to actually do with this
Before you scale real risk behind a backtested setup, check two things: does the sample size clear at least 30 to 50 trades, and does the setup's expectancy meaningfully beat your baseline rather than just being positive? If either answer is no, the honest status isn't "proven" — it's "still testing," and the right move is more data, not more size. The setups that survive going live are, almost without exception, the ones that were given enough trades to prove themselves before a single dollar of real risk was ever put behind them.
