A/B test sample size calculator for two conversion rates

Example output

Sample, not a benchmark: baseline 10 percent and an absolute lift of 2 percentage points, so the second rate is 12 percent. This is a two-sided z-test with unpooled variance, alpha 0.05, and power 0.80. The unrounded count is about 3838.10 and this A/B test sample size calculator rounds up to 3839 visitors per variation, 7678 in total. A pooled-null calculator can print a different integer. This count is not a chance of winning.

How to use it

Enter the current conversion rate as a percent.
Enter the absolute lift in percentage points, not a relative increase.
Read visitors per variation. The count is rounded up.
Do not call a winner before both variations reach that count.

FAQ

Is the result a probability of winning?

No. It is the visitor count from a two-sided z-test with unpooled variance, alpha 0.05 and power 0.80, rounded up to a whole visitor. The minimum detectable effect is an absolute difference in percentage points.

What does an absolute lift of 2 mean?

If the baseline is 10 percent, an absolute lift of 2 percentage points compares 10 percent with 12 percent. It does not mean a 2 percent relative increase.

Why is there no quality score?

A sample size is a count of visitors. This page does not turn that count into a score out of 100.

Sources

Kim, sample size for two independent proportions States the two-sided z approximation with unpooled variance and equal groups, and cites Chow, Shao, and Wang (2003).
Evan Miller's sample size calculator Uses pooled variance under the null, so its integer can differ from the unpooled count on this page.

A/B test sample size calculator