Estimate the GPU requirements of a AI agent using workload assumptions you control. This calculator turns operational inputs into a clear primary estimate plus supporting capacity and planning metrics. Use it for early architecture comparisons, budget reviews, or scenario checks, then replace the defaults with measurements from your own traces and provider terms before making production commitments.
The result rounds a memory requirement up to whole devices and then adds the requested redundancy. It does not benchmark model speed.
Confirm tensor parallelism, supported precision, KV-cache growth, interconnect needs, and actual framework memory use before purchasing or reserving hardware.
For 8 instances using 18 GB each, 72 GB usable per GPU, 15% overhead, and 1 redundant GPU:
No. It sizes GPU count by usable memory; throughput or latency targets may require more devices.
Include weights, KV cache, activations, framework allocations, and expected batching memory.
Drivers, display processes, monitoring, and runtime reservations can reduce memory available to the model.
Yes, but the deployment must support model parallelism; this calculator only aggregates memory capacity.
Enter the number of whole GPUs you intend to keep available for failover or maintenance.
| Variable | Included capacity |
|---|---|
| Model instances | Simultaneously resident processes |
| Memory per instance | Weights, cache, activations, and runtime allocation |
| Usable GPU memory | Deployable capacity per device |
| Redundancy | Whole spare devices added after active sizing |