Plan a random audit of labeled data by estimating the sample needed for a target margin of error. Specify the expected labeling error rate, confidence level, and dataset size to obtain the initial and finite-population-adjusted sample.
p is the expected proportion, e is the desired half-width, and N is the optional dataset size.
Use the main result together with the supporting bounds, counts, or capacity figures. The estimate is only as reliable as the input definitions, sampling process, and operating assumptions.
Stratify the audit when classes, annotators, languages, or data sources have meaningfully different error patterns.
At an expected 10% rate, ±2 percentage-point margin, and 95% confidence, the unadjusted formula requires 865 observations before any finite-population correction.
The calculator uses the normal approximation for a proportion, then applies a finite population correction when a dataset size is supplied.
Use 50% for the most conservative sample under this formula. A reliable pilot estimate can reduce the required sample.
Precision improves with the square root of sample size, so halving the margin generally requires about four times as many observations.
Finite population correction matters when the initial sample is a meaningful share of the entire dataset.
For a simple random-sample interpretation, yes. Stratified or clustered designs need weights or design-effect adjustments.
| Choice | Effect on sample |
|---|---|
| Higher confidence | Increases required sample |
| Smaller margin | Increases required sample substantially |
| Finite dataset | May reduce required sample |