#3139 · AI & Technology Tool

Bayesian Experiment Sample Size Calculator

Estimate an approximate per-variant sample for a Bayesian conversion experiment using a target posterior probability, minimum detectable relative lift, and optional prior strength. The approximation translates the posterior threshold into a normal probability target and shows how prior information affects new-data needs.

Calculator

Enter your assumptions
%
%
%
users
Use zero for a weak, non-informative planning approximation.

How to use this calculator

  1. Enter the baseline measurements or experiment assumptions.
  2. Confirm that each unit matches the label beside the field.
  3. Select Calculate to update the main and secondary results.
  4. Review the interpretation and test a second scenario.

Formula

Effective n ≈ z² × [p₁(1−p₁)+p₂(1−p₂)] ÷ (p₂−p₁)²
New n = max(0, effective n − prior effective n)

What the result means

The estimate is a planning approximation for the sample needed to make the probability of a positive difference reach the chosen threshold at the assumed effect.

Bayesian stopping behavior depends on the full prior, decision rule, and predictive design. Validate important experiments with simulation.

Example calculation

For a 10% baseline, 10% relative lift, 95% posterior target, and no prior effective sample, the normal approximation requires about 5,084 new users per variant.

Tips for better results

  • Use recent, representative measurements rather than optimistic targets.
  • Run a conservative scenario as well as the expected case.
  • Keep units and time periods consistent across every input.
  • Document assumptions so the estimate can be reproduced.
  • Recalculate when traffic, rates, prices, or experiment rules change.

Frequently asked questions

What assumptions does this bayesian experiment sample size estimate use?

It uses the inputs, formula, units, and independence assumptions shown on this page. Change the inputs to match your own system or experiment.

How should I handle traffic or outcome variability?

Use representative averages for planning, preserve headroom where applicable, and test multiple plausible scenarios instead of relying on one point estimate.

Can I use the result as a production or launch guarantee?

No. The result is a planning estimate. Validate it with measured data, monitoring, load tests, or an experiment-design review as appropriate.

What happens if I enter zero or an invalid value?

The calculator checks values required by the formula and displays an error instead of returning NaN or Infinity. Valid zero values remain available where they are meaningful.

Why might another tool return a different answer?

Tools may use different statistical approximations, unit conventions, rounding rules, priors, confidence definitions, or operational headroom assumptions.

Variables and units

VariableUnitMeaning
Posterior targetprobabilityRequired probability that variant exceeds control
Prior effective nusers/variantInformation represented by the prior
New nusers/variantApproximate additional observations

Browse calculator categories

22 category hubs