#3136 · AI & Technology Tool

A/B Experiment Confidence Interval Calculator

Calculate a two-sided confidence interval for the difference between two independent conversion rates. Enter conversions and visitors for control and variant to see the observed absolute lift, relative lift, interval bounds, and a plain-language significance check.

Calculator

Enter your assumptions
events
users
events
users
%

How to use this calculator

  1. Enter the baseline measurements or experiment assumptions.
  2. Confirm that each unit matches the label beside the field.
  3. Select Calculate to update the main and secondary results.
  4. Review the interpretation and test a second scenario.

Formula

CI = (p₂ − p₁) ± zα × √[p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂]

What the result means

The interval gives a range of differences compatible with the data under the normal approximation. If it excludes zero, the observed difference is statistically significant at the selected level.

For very small samples or conversion counts near zero, an exact or score-based interval may be more appropriate.

Example calculation

500/5,000 control conversions and 560/5,000 variant conversions produce rates of 10% and 11.2%. At 95% confidence, the difference is 1.2 percentage points with an interval of roughly −0.01 to 2.41 points.

Tips for better results

  • Use recent, representative measurements rather than optimistic targets.
  • Run a conservative scenario as well as the expected case.
  • Keep units and time periods consistent across every input.
  • Document assumptions so the estimate can be reproduced.
  • Recalculate when traffic, rates, prices, or experiment rules change.

Frequently asked questions

What assumptions does this a/b experiment confidence interval estimate use?

It uses the inputs, formula, units, and independence assumptions shown on this page. Change the inputs to match your own system or experiment.

How should I handle traffic or outcome variability?

Use representative averages for planning, preserve headroom where applicable, and test multiple plausible scenarios instead of relying on one point estimate.

Can I use the result as a production or launch guarantee?

No. The result is a planning estimate. Validate it with measured data, monitoring, load tests, or an experiment-design review as appropriate.

What happens if I enter zero or an invalid value?

The calculator checks values required by the formula and displays an error instead of returning NaN or Infinity. Valid zero values remain available where they are meaningful.

Why might another tool return a different answer?

Tools may use different statistical approximations, unit conventions, rounding rules, priors, confidence definitions, or operational headroom assumptions.

Variables and units

VariableUnitMeaning
p₁, p₂proportionObserved conversions divided by visitors
SEproportionUnpooled standard error
CIpercentage pointsAbsolute difference interval

Browse calculator categories

22 category hubs