Developer

LLM VRAM Calculator

Estimate the GPU VRAM or unified memory a local LLM needs at each quantization, with KV cache for your context length and a fit check.

Loading interactive tool…

Embed this tool on your site

About this tool

The LLM VRAM Calculator estimates how much GPU memory, or unified memory on a Mac, a large language model needs before you download it. Model weights take the number of parameters times the bits per weight, divided by 8: a 70-billion-parameter model is about 140 GB at 16-bit and about 42 GB at a 4.8-bit Q4_K_M quantization. On top of that sits the KV cache, which grows with the context length: 2 × layers × KV heads × head dimension × bytes per value for every token. The calculator adds 10% for runtime buffers, compares the total with the memory you enter, and lists every common quantization side by side. For a mixture-of-experts (MoE) model, enter both the total and the active parameters: every expert must stay in memory, so total parameters decide whether it fits, while the smaller active count decides speed. Add your memory bandwidth to see a theoretical ceiling on tokens per second. Everything runs in your browser.

Why use it

  • Shows weights, KV cache and total separately, so you can see whether the model or the context is the problem.

  • Compares every common quantization at once, from BF16 down to 2-bit, against your memory.

  • Handles mixture-of-experts models correctly: memory follows total parameters, speed follows active ones.

  • Explains every formula on the page, so you can check the numbers yourself.

  • Free, with no sign-up, and nothing you type leaves your browser.

How to use

  1. Pick a preset close to your model, or type its total parameters in billions. For a mixture-of-experts model, also enter the active parameters per token.
  2. Choose a quantization, such as Q4_K_M for a typical GGUF download or BF16 for the original weights.
  3. Set the context length you plan to use and the KV cache precision.
  4. Enter your GPU VRAM, or your Mac's unified memory, to see whether the model fits and how much is left over.
  5. Optionally enter your memory bandwidth for a speed ceiling, and open Model shape to type the layer and head counts from the model's config.json.

When it helps

  • Deciding which GGUF or MLX quantization of a new open-weight model to download.
  • Checking whether a model will fit a 24 GB graphics card or a 64 GB or 128 GB Mac before buying hardware.
  • Working out how much a longer context window adds to memory use.
  • Comparing a dense model with a mixture-of-experts model of similar quality.

Examples

70B dense model at Q4_K_M, 8K context

Input
70B total, 4.8 bits per weight, 80 layers, 8 KV heads, head dimension 128, 16-bit KV cache
Expected result
Weights 42 GB + KV cache 2.68 GB ≈ 49.2 GB total, so it needs a 64 GB machine

816B MoE model with 42B active at 2-bit

Input
816B total, 42B active, 2.6 bits per weight
Expected result
Weights about 265 GB, so it needs a machine with well over 256 GB of memory

Tips

  • Q4_K_M is the usual starting point: about 30% of the 16-bit size, with a small quality loss on most models.
  • If a model barely fits, shorten the context or use an 8-bit KV cache before dropping to a lower-bit quantization.
  • Leave room for your operating system and other apps; on a Mac the GPU cannot use all of unified memory.
  • Download size on disk is close to the weights figure for the same quantization.

Frequently Asked Questions

How much VRAM do I need to run an LLM?
Roughly the parameter count in billions times the bits per weight, divided by 8, in gigabytes, plus the KV cache and about 10% overhead. An 8B model at Q4_K_M needs about 5 GB for weights, while a 70B model needs about 42 GB.
Why does a mixture-of-experts model need so much memory if only a few experts run?
Each token can be routed to different experts, so all of them must be loaded and ready. Memory follows the total parameter count; only speed benefits from the smaller number of active parameters.
What is the KV cache?
It stores the attention keys and values for every token already in the context, so the model does not recompute them. It grows linearly with context length, which is why long contexts can need many extra gigabytes.
What do Q4_K_M, Q8_0 and NVFP4 mean?
They are quantization formats that store weights in fewer bits. Q8_0 uses about 8.5 bits per weight, Q4_K_M about 4.8 and NVFP4 about 4.5, counting the scale values stored alongside the weights. Fewer bits use less memory but can reduce quality.
Can I run a model that does not fit in my GPU?
Some runtimes can offload layers to system RAM or disk, but generation then slows down a lot, because those layers are read over a much slower connection. The calculator assumes the whole model sits in GPU or unified memory.
How accurate is the speed estimate?
It is an upper bound: memory bandwidth divided by the bytes read for each token. Real speeds are lower because of compute limits, software overhead and long contexts.

Explore the category

What GPU Do I Have? See which graphics card your browser uses, whether it is integrated or dedicated, and whether WebGPU works. No download or sign-up. GB to GiB Converter Convert bits and bytes with SI decimal prefixes or IEC binary kibibytes and tebibytes. JSON Formatter Format, validate and minify JSON File Hash Multi (MD5, SHA-1, SHA-256, SHA-512) Compute MD5, SHA-1, SHA-256, and SHA-512 of any file in your browser Percentage Difference Calculator Find the percent difference between two numbers, plus the percent change from A to B and from B to A, with each formula worked out. Unit Converter Convert length, weight, temperature and more
Can You Run Muse Spark Open Locally? Memory at Every Quantization Muse Spark Open has 816 billion parameters and needs about 260 to 300 GB even at 2-bit. Memory for every quantization, and which machines ca Oct 2026 What Is Mixture of Experts? Why MoE Models Are Fast but Huge A mixture-of-experts model uses only a few of its expert networks per token. How routing works, total vs active parameters, and why MoE stil Oct 2026 What Is Unified Memory? How It Differs From VRAM, and How Much You Need Unified memory is one pool of RAM shared by the CPU and GPU. How it compares with VRAM, why it lets laptops run big AI models, and how much Oct 2026

Glossary

Quantization
Storing model weights with fewer bits, such as 4 instead of 16, to cut memory use.
Mixture of experts (MoE)
A model design where a router sends each token to a few expert sub-networks out of many.
Unified memory
Memory shared by the CPU and GPU, as on Apple silicon Macs, which lets the GPU use most of system RAM.
A LITTLE HELP. A LOT DONE.
Good tools.
Great flow.

Your shortcut to the everyday. Create, convert, calculate,
and get back to what you love.

✓ Free to use✓ No sign-up✓ 500+ possibilities