The honest answer is "until each version has enough visitors, and never less than a full week." Here is how long real tests on Stellar ran, what traffic they needed, and a formula you can use to plan your own.
Data through 13 Nov 2026 · window 28 Feb 2025 – 13 Nov 2026 · n = 450 tests, 125 accounts.
Runtime is the number of days between a test's first and last visitor.
| Runtime | Share of qualified tests | Tests (accounts) |
|---|---|---|
| 7–13 days | 6.4% | 29 (19) |
| 14–27 days | 16.4% | 74 (38) |
| 28–55 days | 25.1% | 113 (60) |
| 56–111 days | 25.1% | 113 (57) |
| 112 days or more | 26.9% | 121 (52) |
Fewer than one test in four finished in under four weeks. More than half ran eight weeks or longer.
Data through 13 Nov 2026 · window 28 Feb 2025 – 13 Nov 2026 · n = 450.
Not by itself. This is the share of tests in each runtime group where one version reached at least 95% probability of being best (our recalculation):
| Runtime | Reached a clear result |
|---|---|
| 7–13 days | 31.0% |
| 14–27 days | 20.3% |
| 28–55 days | 25.7% |
| 56–111 days | 29.2% |
| 112 days or more | 27.3% |
Across every runtime group, roughly one test in four to one in three reached a clear result. Tests that end fast are often the ones with a big, obvious difference. Tests that run for months often keep running because the difference is small. Extra weeks only help if they add the visitors your planned effect size needs. If the difference you're chasing is tiny, more time mostly buys more uncertainty.
Data through 13 Nov 2026 · window 28 Feb 2025 – 13 Nov 2026 · n = 450.
The median qualified test collected 39.5 visitors per version per day (p25 17.4, p75 85.1). At that pace, 2,016 visitors per version takes about seven weeks.
Grouped by how many visitors each version collected by the end of the test:
| Visitors per version (median of the test) | Tests (accounts) | Median runtime (p25–p75) | Reached a clear result |
|---|---|---|---|
| 300–999 | 132 (70) | 33 days (19–71) | 18.9% |
| 1,000–4,999 | 197 (74) | 59 days (34–106) | 32.0% |
| 5,000–19,999 | 85 (39) | 109 days (54–161) | 21.2% |
| 20,000 or more | 36 (16) | 83.5 days (51–189.5) | 36.1% |
Tests that stopped at under 1,000 visitors per version reached a clear result least often (18.9%). Small samples can only detect big differences.
Data through 13 Nov 2026 · window 28 Feb 2025 – 13 Nov 2026 · n = 450.
All examples use 95% confidence, 80% power, two versions and 1,000 visitors a day entering the test, unless noted.
| Baseline | Smallest lift to detect | Visitors per version | Raw days | Plan for |
|---|---|---|---|---|
| 3% | +20% | 13,914 | 28 | 4 weeks |
| 3% | +10% | 53,211 | 107 | 16 weeks |
| 5% | +20% | 8,158 | 17 | 3 weeks |
| 3% | +20%, three versions | 16,850 | 51 | 8 weeks |
| 10% | +10% | 14,751 | 30 | 5 weeks |
The three-version row adjusts confidence for two comparisons against the control. That's one reason extra versions slow tests down.
The sample size and duration calculator turns your baseline, lift and traffic into visitors per version and whole weeks.
Stellar shows each version's probability of beating the control, using Bayesian statistics, so you know whether to keep going.
Stellar Autopilot needs 95% probability, at least 7 days and 300+ visitors per version before it keeps a winner. See Autopilot
If a test runs long with no winner, see whether visitors even reach the element you changed. Hotjar and Clarity with A/B testing
Pricing: Sandbox is $0 for up to 5k monthly tracked users. Plus is $29/mo billed annually for up to 25k, and Pro is $99/mo billed annually for up to 50k.
How long should an A/B test run?
Until each version reaches its planned sample size, rounded up to whole weeks, with at least 7 days. In 450 qualified tests on Stellar (28 Feb 2025 – 13 Nov 2026), the median ran 57 days.
Is two weeks enough for an A/B test?
Only if your traffic gets each version to its planned sample size in two weeks. In our data, 22.8% of qualified tests finished in under four weeks (6.4% under two weeks, 16.4% in two to four).
What is the minimum duration for an A/B test?
At least 7 days, so every weekday is represented. Stellar Autopilot also requires 300+ visitors per version. Most pages need more than both minimums to detect a realistic lift.
Can I stop an A/B test early if it's winning?
Checking often is fine. Deciding on an early lead is risky. Wait for your planned sample, whole weeks and a clear probability. In our data only about one test in six ended with a variant clearly beating the control.
Why is my A/B test taking so long?
Usually low traffic, a low baseline conversion rate, a small difference between versions, or too many versions. The median test on Stellar collected 39.5 visitors per version per day.
How many visitors do I need for an A/B test?
It depends on your baseline and the lift you want to detect. At a 3% baseline, detecting a +20% lift takes about 13,914 visitors per version (95% confidence, 80% power). The median qualified Stellar test had 2,016 visitors per version.