#3134 · AI & Technology Tool

A/B Experiment Sample Size Calculator

Estimate the visitors required in each variant of a two-group conversion experiment. The calculation uses a two-sided normal approximation with equal allocation and accounts for the baseline conversion rate, minimum detectable relative lift, confidence level, and desired power.

Calculator

Enter your assumptions
%
%
%
%

How to use this calculator

  1. Enter the baseline measurements or experiment assumptions.
  2. Confirm that each unit matches the label beside the field.
  3. Select Calculate to update the main and secondary results.
  4. Review the interpretation and test a second scenario.

Formula

n = [zα·√(2p̄(1−p̄)) + zβ·√(p₁(1−p₁)+p₂(1−p₂))]² ÷ (p₂−p₁)²

This is a two-sided, equal-allocation normal approximation.

What the result means

The main result is the minimum rounded-up sample for each variant. Smaller effects and higher confidence or power require more observations.

This fixed-horizon estimate does not correct for repeated peeking, multiple metrics, clustering, or traffic imbalance.

Example calculation

At a 10% baseline, 10% relative lift (11% variant rate), 95% confidence, and 80% power, the calculator requires 14,751 users per variant under this approximation.

Tips for better results

  • Use recent, representative measurements rather than optimistic targets.
  • Run a conservative scenario as well as the expected case.
  • Keep units and time periods consistent across every input.
  • Document assumptions so the estimate can be reproduced.
  • Recalculate when traffic, rates, prices, or experiment rules change.

Frequently asked questions

What assumptions does this a/b experiment sample size estimate use?

It uses the inputs, formula, units, and independence assumptions shown on this page. Change the inputs to match your own system or experiment.

How should I handle traffic or outcome variability?

Use representative averages for planning, preserve headroom where applicable, and test multiple plausible scenarios instead of relying on one point estimate.

Can I use the result as a production or launch guarantee?

No. The result is a planning estimate. Validate it with measured data, monitoring, load tests, or an experiment-design review as appropriate.

What happens if I enter zero or an invalid value?

The calculator checks values required by the formula and displays an error instead of returning NaN or Infinity. Valid zero values remain available where they are meaningful.

Why might another tool return a different answer?

Tools may use different statistical approximations, unit conventions, rounding rules, priors, confidence definitions, or operational headroom assumptions.

Variables and units

VariableUnitMeaning
Baseline p₁proportionControl conversion probability
Alternative p₂proportionBaseline × (1 + relative lift)
nusers/variantEqual-allocation fixed-horizon sample

Browse calculator categories

22 category hubs