Estimate expected false positives and missed effects across a portfolio of A/B experiments. Enter the number of tests, significance threshold, power, and expected share of true effects to separate Type I and Type II error counts.
The result summarizes long-run expected errors across repeated experiments, not a guarantee for a particular test.
The true-effect share is an assumption. Dependence between tests and selective reporting can change realized error rates.
Across 100 experiments with α = 5%, 80% power, and 20% true effects, the long-run expectation is 4 false positives and 4 false negatives.
It uses the inputs, formula, units, and independence assumptions shown on this page. Change the inputs to match your own system or experiment.
Use representative averages for planning, preserve headroom where applicable, and test multiple plausible scenarios instead of relying on one point estimate.
No. The result is a planning estimate. Validate it with measured data, monitoring, load tests, or an experiment-design review as appropriate.
The calculator checks values required by the formula and displays an error instead of returning NaN or Infinity. Valid zero values remain available where they are meaningful.
Tools may use different statistical approximations, unit conventions, rounding rules, priors, confidence definitions, or operational headroom assumptions.
| Variable | Unit | Meaning |
|---|---|---|
| α | probability | False-positive rate when no effect exists |
| 1 − power | probability | False-negative rate for detectable effects |
| True-effect share | proportion | Planning assumption across tests |