AB Test Size Calculator
Estimate the required sample size for a statistically sound A/B test. Enter your baseline conversion rate, minimum detectable uplift, confidence level, statistical power, and daily traffic to calculate visitors needed per variation and projected test duration.
Example: if your current page converts at 5%, enter 5.
Relative lift you want to detect, such as 10% over baseline.
Use average eligible daily traffic, not total site traffic.
For a balanced A/B test, keep this at 50% per variation.
Your results will appear here
Enter your assumptions and click calculate to estimate sample size per variation, total sample, expected conversions, and approximate test duration.
Required sample size by uplift scenario
Chart updates automatically after each calculation. Smaller detectable uplifts require larger samples.
How an AB test size calculator helps you run more reliable experiments
An AB test size calculator helps teams answer one of the most important experiment design questions before any traffic is exposed to a new variant: how many users do we need to observe a real difference with confidence? Without a sample size estimate, marketers, product managers, growth analysts, and UX researchers often stop tests too early, celebrate noise as if it were insight, or miss meaningful improvements because the experiment never had enough statistical power.
In a classic A/B test, you compare two versions of a page, flow, email, or feature. Version A is the control, and Version B is the treatment. If your outcome is binary, such as converting or not converting, the key metric is usually a conversion rate. The challenge is that conversion rates naturally bounce around due to randomness. A sample size calculator gives you a structured way to estimate how much traffic is needed to separate random variation from a likely true effect.
This calculator uses a two-sample proportion framework, which is appropriate for many common conversion optimization tests. It incorporates your baseline conversion rate, desired minimum detectable effect, confidence level, and statistical power. It can also estimate test duration based on your available traffic, which is often the operational bottleneck for experimentation programs.
What the calculator inputs mean
1. Baseline conversion rate
Your baseline conversion rate is the current or historical probability that a user converts under the control experience. If 5 out of every 100 users convert, your baseline is 5%. This number matters because the variance of a proportion depends on the proportion itself. In practical terms, a test at a 2% baseline usually needs a different sample size than a test at a 20% baseline, even when the targeted uplift is expressed as a percentage.
2. Minimum detectable uplift
The minimum detectable uplift is the smallest relative improvement worth detecting. If your baseline rate is 5% and you enter a 10% uplift, the calculator assumes the treatment conversion rate you want to detect is 5.5%. Smaller lifts are harder to detect because the absolute difference is tiny, so sample sizes increase quickly as your minimum detectable effect shrinks.
3. Confidence level
Confidence level is tied to your false positive risk, also called Type I error. A 95% confidence level is a standard choice in experimentation because it means your threshold for declaring significance is stricter than 90% but less demanding than 99%. Higher confidence levels reduce the chance of false positives, but they also increase required sample size.
4. Statistical power
Power is the probability that your experiment will detect a real effect of the size you care about. A common benchmark is 80% power. If your test truly has the uplift you specified, an 80% powered experiment has an 80% chance of finding a statistically significant result. Higher power reduces false negatives, but it also requires more traffic.
5. Daily traffic and split
Even if a test is statistically well designed, it still has to be practical. Traffic determines how long it will take to reach the required sample size. A 50/50 traffic split usually minimizes time for a two-variant A/B test, assuming equal business risk across versions.
Key idea: If you want to detect very small uplifts with high confidence and high power, your required sample can become very large. This is why test planning is as important as test execution.
Why sample size matters so much
Many failed experimentation programs are not failing because the team has poor ideas. They fail because the team has weak measurement discipline. An underpowered test can easily produce one of two bad outcomes. First, it may show a dramatic apparent winner that disappears once the sample grows. Second, it may fail to detect a genuinely useful improvement, leading the team to reject a good idea. Both outcomes waste time and can distort product strategy.
A sound sample size estimate helps you avoid peeking bias and prevents stopping a test after only a handful of conversions. It also lets you decide whether a test is worth running at all. If your site traffic is low and you want to detect a 2% relative uplift, the test may take months. In that case, you might choose a higher impact hypothesis, a more sensitive metric, or a broader redesign with a larger expected effect.
Reference table: common confidence and power values
The z-scores below are standard statistical constants used in sample size calculations for normal approximations.
| Parameter | Level | Z value | Meaning in practice |
|---|---|---|---|
| Confidence | 90% | 1.645 | Faster tests, higher false positive tolerance |
| Confidence | 95% | 1.960 | Most common balance for business experiments |
| Confidence | 99% | 2.576 | Very strict, often much larger sample needed |
| Power | 80% | 0.842 | Standard benchmark for many A/B tests |
| Power | 90% | 1.282 | Lower false negative risk, slower to complete |
| Power | 95% | 1.645 | Stringent, often reserved for critical decisions |
Illustrative sample size comparisons
The table below shows how dramatically the required sample per variant can change as your baseline rate and minimum detectable uplift change. These figures are illustrative for a two-variant test using 95% confidence and 80% power.
| Baseline conversion | Relative uplift | Expected treatment rate | Approx. sample per variant |
|---|---|---|---|
| 2.0% | 10% | 2.2% | About 77,800 users |
| 5.0% | 10% | 5.5% | About 31,300 users |
| 10.0% | 10% | 11.0% | About 14,700 users |
| 5.0% | 20% | 6.0% | About 8,200 users |
| 5.0% | 5% | 5.25% | About 124,400 users |
How to interpret your calculator result
When you click calculate, the tool returns sample size per variant, total sample size, expected conversions under control and treatment assumptions, and an estimated duration in days. The most important output is usually the sample required per variant, because that tells you what each side of the test needs to observe before you can reliably judge the outcome.
- Sample per variant: the number of users needed in both control and treatment groups.
- Total sample: the sum across both variants.
- Expected treatment conversion rate: your baseline adjusted by the uplift assumption.
- Estimated duration: how long it may take to reach the required sample if traffic remains stable.
Keep in mind that duration estimates are only as good as your traffic estimate. If traffic has strong weekday and weekend swings, or if only a subset of users is eligible for the experiment, use the average traffic that truly enters the test.
Common mistakes when using an AB test size calculator
- Using total site traffic instead of eligible traffic. If only 30% of visitors reach the tested page, using all site traffic will understate test duration.
- Choosing an unrealistic minimum detectable effect. Many teams want to detect a 2% relative lift, but their traffic only supports larger changes in a reasonable timeframe.
- Stopping early after a promising spike. Early wins often regress as more data arrives.
- Ignoring seasonality and novelty effects. A test that runs through a holiday or campaign launch may behave differently from a normal week.
- Running too many variants with too little traffic. More variants spread traffic thinner and lengthen duration.
Practical advice for choosing a minimum detectable effect
A good minimum detectable effect is not just a statistical preference. It should reflect business value. Suppose your signup rate is 4%, and each signup is worth a known amount of revenue or predicted lifetime value. If a 5% relative uplift creates meaningful impact over a quarter, that may be worth planning for. If not, your team might prioritize hypotheses with the potential for larger gains.
In general, lower traffic sites often need to focus on higher impact experiments. Higher traffic sites can afford to test smaller optimizations because they can collect enough data faster. This is one reason large consumer platforms can optimize aggressively, while smaller businesses should concentrate on bigger funnel opportunities, messaging shifts, pricing tests, or major UX bottlenecks.
When to trust, and when to question, the result
This type of calculator is very useful, but no calculator can replace sound experimental judgment. The result is most trustworthy when your metric is binary, your users are independently assigned, traffic is stable, and the test is truly randomized. If your environment includes heavy user heterogeneity, repeated exposure, strong novelty effects, clustered data, or multiple primary metrics, you may need a more advanced design or a specialist review.
For example, product experiments with returning users can be more complex than page-level tests because users may visit multiple times, and outcomes can depend on prior exposure. Likewise, subscription or revenue experiments can require extra care if the primary metric is not a simple proportion.
Authoritative statistical references
If you want deeper technical guidance on hypothesis testing, power, and sample size, these sources are excellent starting points:
- NIST Engineering Statistics Handbook
- Penn State STAT Online
- University of California, Berkeley Statistics
Final takeaway
An AB test size calculator is one of the simplest ways to improve the rigor of your experimentation process. It helps you align stakeholders around realistic expectations, avoid underpowered tests, and balance statistical confidence with operational constraints. If your result says the test will take too long, that is not a failure. It is a useful planning signal. You can refine the hypothesis, choose a larger expected effect, improve targeting, or wait until traffic is sufficient. Good experimentation is not just about creativity. It is also about respecting the math that turns observations into trustworthy decisions.