Free CRO tool
A/B Test Duration Calculator
Know exactly how many days to run your test, before you start. Assumes 95% confidence and 80% statistical power.
Your results
- Estimated test duration
- —
- Significance
- —
- Statistical power
- —
- Target conv. rate
- —
- Visitors / variation / day
- —
Not sure what to test next? Run a free CRO audit
How to use this calculator
- Enter daily visitors. Use the total combined visitors entering the test per day, both control and variation traffic combined. Find this in your analytics under your test page's daily sessions.
- Set your baseline rate. Enter your control page's current conversion rate as a percentage. For example, if 150 out of 5,000 visitors convert, enter 3.0.
- Choose your MDE. The minimum detectable effect is the smallest relative improvement worth detecting. A 15% MDE on a 2% baseline means you want to detect when the variation reaches 2.3%. Smaller MDEs need more time.
Why test duration matters
One of the most damaging mistakes in A/B testing is stopping a test as soon as you see a "winner." This practice, known as peeking, dramatically inflates your false positive rate. When you stop early, you're essentially looking at a sample that happens to show a difference by chance.
Running tests to their predetermined duration (calculated before the test starts) is called fixed-horizon testing and is the gold standard for reliable A/B test results. It ensures your statistical guarantees are valid and your shipping decisions are based on real signal.
Even if the math suggests you need fewer than 7 days, always run tests for at least one full week to capture day-of-week behavioral variation. Users behave very differently on Mondays versus Saturdays, and a test that runs Monday through Wednesday will over-represent weekday traffic patterns.
Frequently asked questions
How long should an A/B test run?
An A/B test should run until it reaches the required sample size, which depends on your daily traffic, baseline conversion rate, and the minimum effect you want to detect. Most tests need at least 1–4 weeks. Never stop a test early just because results look promising; this leads to false positives.
What is a minimum detectable effect (MDE)?
The MDE is the smallest relative improvement you want your test to reliably detect. A 15% MDE on a 2% baseline means detecting when the variation reaches 2.3% conversion. Smaller MDEs require larger samples and longer tests. For most tests, an MDE of 10–20% is practical; detecting smaller effects requires very high traffic volumes.
What does 95% statistical significance mean?
95% significance means there's only a 5% probability (p < 0.05) that the observed difference happened due to chance. It does not mean you're 95% confident in the effect size or that the variation will perform the same way in production. It's a threshold for decision-making, not a guarantee.
What is 80% statistical power?
Statistical power is your test's ability to correctly detect a true effect when one exists. 80% power means that if your variation really does improve conversions by your MDE, your test has an 80% chance of detecting it as significant. The remaining 20% are false negatives: real effects you miss because your sample was too small.
Can I run a test for less than 7 days?
Even if the formula suggests fewer days, always run tests for at least 7 days to account for day-of-week effects. User behavior differs significantly between weekdays and weekends. A test that runs Thursday through Sunday will over-represent weekend visitors and produce biased results that don't hold during the full week.
The formula
- n
- Visitors each variation needs (two-proportion sample size).
- Zα/2
- 1.96 (95% significance).
- Zβ
- 0.8416 (80% power).
- p₁, p₂
- Baseline conversion rate and p₁ × (1 + MDE/100).
- ⌈ x ⌉
- Rounded up to the next whole number.
The result is in days, with weeks shown once a test runs longer than a week. Even when the maths allows fewer than 7 days, run at least one full week so every day of the week is represented.