Estimate sustained throughput for LLM batch processing using workload inputs you can replace with measurements from your own model and serving environment. The calculator shows the primary planning result together with supporting capacity or cost figures, so you can compare demand, operating assumptions, and available resources. It runs entirely in your browser and does not send workload data to a server.
Record throughput = workers × utilization ÷ seconds per record. Token throughput = record throughput × tokens per record.
Every rate is applied in the unit shown beside its input. Values are calculated without intermediate rounding; displayed results are rounded for readability.
The main result is a planning estimate of sustained throughput for the stated batch processing assumptions. Supporting values expose the capacity, reserve, time, or cost components behind that estimate.
This calculator is an engineering estimate, not a guarantee. Benchmark the exact model, hardware, provider, and prompt distribution before production sizing or procurement.
32 workers at 80% utilization with 2 seconds per record process 12.8 records/second, or 12,800 tokens/second at 1,000 tokens per record.
Use observed averages from the same model, hardware, prompt mix, and serving configuration whenever possible. Planning inputs are only as reliable as the measurements supplied.
Only overhead represented by the visible inputs is included. Add a conservative allowance or use measured end-to-end values when queueing, model loading, storage, or orchestration is material.
Yes for an initial budget, but also test separate high-volume and long-context scenarios because averages can hide peak resource demand.
LLM performance and pricing depend on model size, hardware, precision, batching, provider, and prompt mix, so a universal benchmark would be misleading.
Run a representative load test, compare observed totals with the estimate, and update the inputs before making a production capacity or purchasing decision.
| Variable | How to use it |
|---|---|
| Records | Total batch records. |
| Tokens per record | Average input plus output tokens. |
| Parallel workers | Active processing slots. |
| Seconds per record | Average processing duration. |
| Utilization | Sustained worker utilization. |