LLM VRAM calculator
How much GPU memory a model needs for its weights and KV cache, for your context length and number of users, and which GPUs fit. Runs in your browser.
28 GBestimated GPU memory
| Model weights | 18 GB |
|---|---|
| KV cache | 8.6 GB |
| Runtime overhead | 1.5 GB |
GPUs that fit
| GPU memory | Examples | Cards needed |
|---|---|---|
| 16 GB | RTX 4080 / 5080 class | 2 |
| 24 GB | RTX 4090, L4 class | 2 |
| 32 GB | RTX 5090 class | 1 |
| 48 GB | RTX 6000 Ada, L40S class | 1 |
| 80 GB | A100 / H100 80 GB class | 1 |
| 96 GB | RTX PRO 6000 Blackwell class | 1 |
| 128 GB unified | DGX Spark; Mac with 128 GB | 1 |
| 141 GB | H200 class | 1 |
Unified memory (DGX Spark, Mac) holds the model but generates more slowly than datacenter GPUs; check tokens per second, not only fit.
How the estimate works
Weights = parameters × bytes per parameter: 2 bytes at 16-bit, 1 byte at 8-bit and about 0.56 bytes at 4-bit, because 4-bit formats also store scales.
KV cache = 2 × layers × KV heads × head dimension × bytes per value × context length × concurrent requests. Models with grouped-query attention (few KV heads) need far less cache than older designs.
Runtime overhead covers the CUDA context, activations and allocator slack: about 8% of the weights, at least 1.5 GB. Multi-GPU setups keep about 10% of each card free.
GPU memory by model size
Weights only, plus the KV cache for one 8,192-token request at 16-bit. Values from the formulas above.
| Model class | 16-bit | 8-bit | 4-bit | KV cache, 8k context |
|---|---|---|---|---|
| 8B (Llama 3.1 8B class) | 16 GB | 8.0 GB | 4.5 GB | 1.1 GB |
| 14B (Qwen2.5 14B class) | 30 GB | 15 GB | 8.3 GB | 1.6 GB |
| 32B (Qwen2.5 32B class) | 65 GB | 33 GB | 18 GB | 2.1 GB |
| 70B (Llama 3.1 70B class) | 141 GB | 71 GB | 40 GB | 2.7 GB |
| 120B mixture-of-experts (gpt-oss-120b class) | 234 GB | 117 GB | 66 GB | 0.6 GB |
| 405B (Llama 3.1 405B class) | 810 GB | 405 GB | 228 GB | 4.2 GB |
Mixture-of-experts models such as gpt-oss-120b must hold every expert in memory, even though only a few run per token. They need memory like a large model but generate at the speed of a smaller one.
Questions
How much VRAM do I need to run an LLM?
Roughly the model size in billions of parameters times the bytes per parameter: about 2 GB per billion parameters at 16-bit, 1 GB at 8-bit and a little over 0.5 GB at 4-bit. Add the KV cache, which grows with context length and the number of requests served at once, plus 1.5 GB or more for the runtime.
What is the KV cache?
While a model generates text it keeps the keys and values of every previous token in GPU memory so it does not recompute them. The cache grows linearly with the context length and with the number of concurrent requests. For long documents or many users it can take more memory than the model weights.
Does 4-bit quantization hurt quality?
Modern 4-bit formats (AWQ, GPTQ, GGUF Q4, MXFP4) lose little quality on most tasks for models of 14B parameters and up; small models lose more. Test the quantized model on your own examples before you commit to hardware sized for it.
Can I split a model across several GPUs?
Yes. Serving engines such as vLLM split the weights and KV cache across GPUs in one server (tensor parallelism). Each card needs some free memory, which the calculator accounts for by using about 90% of each GPU.
How accurate is this calculator?
It is a first estimate. Real usage depends on the serving engine, its memory settings, batch sizes and model details such as sliding-window attention. Use it to shortlist hardware, then measure with your own prompts before buying.
Next: compare what that hardware costs with API pricing in the LLM cost calculator, or read which hardware to buy for local LLMs.