Estimate GPU count for LLM online inference using workload inputs you can replace with measurements from your own model and serving environment. The calculator shows the primary planning result together with supporting capacity or cost figures, so you can compare demand, operating assumptions, and available resources. It runs entirely in your browser and does not send workload data to a server.
Required GPUs = ceiling[required tokens/second × (1 + redundancy) ÷ (per-GPU tokens/second × utilization)].
Every rate is applied in the unit shown beside its input. Values are calculated without intermediate rounding; displayed results are rounded for readability.
The main result is a planning estimate of GPU count for the stated online inference assumptions. Supporting values expose the capacity, reserve, time, or cost components behind that estimate.
This calculator is an engineering estimate, not a guarantee. Benchmark the exact model, hardware, provider, and prompt distribution before production sizing or procurement.
At 5,000 required tokens/second, 850 tokens/second per GPU, 75% utilization, and 20% redundancy: ceil(6,000 ÷ 637.5) = 10 GPUs.
Use observed averages from the same model, hardware, prompt mix, and serving configuration whenever possible. Planning inputs are only as reliable as the measurements supplied.
Only overhead represented by the visible inputs is included. Add a conservative allowance or use measured end-to-end values when queueing, model loading, storage, or orchestration is material.
Yes for an initial budget, but also test separate high-volume and long-context scenarios because averages can hide peak resource demand.
LLM performance and pricing depend on model size, hardware, precision, batching, provider, and prompt mix, so a universal benchmark would be misleading.
Run a representative load test, compare observed totals with the estimate, and update the inputs before making a production capacity or purchasing decision.
| Variable | How to use it |
|---|---|
| Required token throughput | Combined input and output token demand. |
| Measured throughput per GPU | Benchmark from the intended model, hardware, and precision. |
| Usable utilization | Fraction of benchmark capacity available to the workload. |
| Redundancy allowance | Extra capacity for failures, spikes, or maintenance. |