AI Agent Teams Are Becoming a Distributed Systems Problem

A single AI agent can be evaluated much like a skilled employee: give it a task, inspect its output, measure accuracy, cost and time. An agent team is different. Once multiple agents share a codebase, tool permissions, task state or a marketplace, the important unit is no longer the model. It is the system formed by their interactions.

That shift matters because the industry is beginning to build systems in which agents delegate, review, negotiate and act in parallel. The upside is obvious. A team can search more code, test more hypotheses and cover more ground than one agent working serially. The risk is less obvious: coordination becomes its own technical discipline.

Anthropic’s Frontier Red Team found in research on emerging multiagent systems that coordinated agents can improve vulnerability discovery, but that agents operating together can also converge on bad judgments, develop incompatible plans, abandon conflicting code changes and, when goals differ, sabotage one another. The implication is that the familiar question, “How capable is the model?” is becoming insufficient. The harder question is, “What happens when capable models share a world?”

“In the future, we predict that this sort of specialization and coordination will dominate over uncoordinated brute-force search.”

The distributed-systems analogy is more than a metaphor

A modern agent is already more than a language model. Anthropic describes agents as systems made up of a model, a harness, tools and an environment, with each layer contributing both capability and a potential point of failure. A multi-agent deployment repeats that stack across several actors, then adds communication channels, shared memory, permissions, queues and rules for resolving disagreements.

That is recognizably distributed software.

The model is analogous to a service. Prompts and role descriptions function like service contracts. Messages, pull requests and tool calls are the network traffic. Shared repositories, databases and task boards are the stateful infrastructure. Retries can create duplicate work. A manager agent is an orchestrator. An evaluator agent is a validation layer. Human approval becomes an escalation path.

The comparison is useful because distributed systems have long taught engineers that individually correct components can collectively fail. Two agents may both reasonably conclude that they own a task. A worker may act on stale information because another agent changed the plan. A reviewer may approve a patch that passes its local tests while breaking the wider product. A swarm can also produce a subtler failure: all agents can repeat the same flawed assumption, creating the appearance of confirmation without genuine independent judgment.

Anthropic’s experiments illustrate both sides. In a vulnerability-search test, it gave 45 agents separate virtual machines, a shared forum and access to 15 open-source projects, then used another agent as an arbiter of submitted findings. The coordinated swarm found 266 vulnerabilities in one run, compared with 21 found by independent parallel agents. But the comparison also had a crucial qualification: the coordinated team used more tokens and discovered many findings outside the core directories where the independent agents had been directed to search.

The lesson is not that swarms always win. It is that parallel exploration works when tasks are genuinely separable, and that the measurement must distinguish broader search from better coordination.

Three organizational designs, three distinct risks

The simplest multi-agent architecture is manager-worker. A manager decomposes a goal, assigns work, gathers outputs and decides what to ship. This creates accountability and limits uncontrolled peer interaction. It is often the most sensible design when work can be divided into bounded assignments, such as researching competitors, writing test cases or checking a set of files.

Its weakness is the manager itself. If it frames the task poorly, every worker can optimize for the wrong objective. It can also become a throughput bottleneck, especially when workers need to negotiate dependencies rather than simply return answers.

Peer-to-peer systems promise greater flexibility. Agents can discover work, form temporary teams and exchange information directly. That can be valuable in open-ended exploration, where a rigid plan would prematurely constrain the search. But it also creates the classic problems of distributed coordination: Who owns a decision? Which version of shared state is authoritative? When do agents stop debating and act?

Anthropic’s code-building simulations show why those questions matter. The company tested agent swarms building text-based fantasy games with shared forums and repositories. It varied team size, model generation and prompts, including prescribed roles and a CEO-style hierarchy. Those prompt changes did not materially solve the problem. Earlier models opened many pull requests but merged few, while newer models often avoided conflict by preserving strong file ownership rather than collaborating deeply.

A third design is evaluator-optimizer. One group generates candidate plans, code or decisions. Another checks them against specifications, tests or safety criteria. This is often more robust than asking every agent to be both creator and judge. It separates incentives and creates an explicit arbitration point.

But evaluators are not neutral by default. If a reviewer shares the same model, context and assumptions as the workers, it may reproduce their mistake. The value of review depends on meaningful diversity: different evidence, different tools, adversarial tests or independent task formulations.

More agents can reduce coherence

The strongest case for multi-agent teams is specialization. One agent can inspect authentication logic, another memory-safety issues and another deployment configuration. A good team can maintain a broader search frontier than a single agent with a fixed sequence of attention.

The strongest case against indiscriminate use is that coordination has a price. TMLS AI Research argues that teams are frequently over-applied, creating token costs and management overhead that outweigh any gain when one agent could complete the work. Its practical recommendation is demanding but useful: begin with a single agent, then add agents only when subtasks are truly independent.

That is the critical discipline missing from much agent design. Parallelism is not automatically productive. If two workers need the same evolving context, a shared tool or a single final decision, their communication and reconciliation costs can dominate their output.

The test is simple. If splitting a task creates more handoffs than completed work, hierarchy or serial execution is likely better. If work units are independent, cheaply verifiable and can be combined through a stable interface, parallel teams become more attractive.

Contracts matter more than clever prompts

The operational answer is to treat each agent handoff as an API contract. A worker should know its scope, what evidence it must return, which files or records it may modify, when it must ask for clarification and what it is forbidden to do. A manager should receive structured artifacts, not opaque prose that must be interpreted afresh.

Shared-state controls are equally important. Teams need task ownership, versioned artifacts, idempotent tool calls and conflict detection before changes reach production. For consequential actions, they also need containment: limited credentials, spending limits, approval gates and the ability to stop or roll back a group of agents without losing the audit trail.

Observability must change, too. Logging one model’s chain of tool calls is not enough. Teams need traces that show which agent delegated what, which state version it saw, which tools it used, what changed and why an arbiter accepted or rejected the result. The critical metric is not merely task completion. It is the rate of successful outcomes after accounting for duplicate work, conflicts, reversals, oversight burden and recovery time.

Google DeepMind’s response suggests that this is increasingly viewed as a field-level issue, not just a product feature. Google DeepMind and its partners announced a funding call of up to $10 million for research on multi-agent testbeds, agent networks, infrastructure, oversight and collective risks. The program specifically highlights simulated marketplaces, multi-organization workflows, identity, reputation and methods for monitoring deployed agent populations.

The evaluation gap is now the strategic gap

Single-agent benchmarks ask whether an agent can write code, find information or use a tool. Multi-agent evaluations must ask whether a system can sustain a shared objective under pressure.

That means testing stale state, conflicting instructions, partial tool outages, duplicate assignments, malicious messages, uneven permissions and incentives that reward local output at the expense of the final result. It also means comparing architectures fairly. A swarm that uses four times the compute to search a wider area has not necessarily demonstrated superior coordination.

LessWrong commentator Shunk noted a related concern about Anthropic’s vulnerability experiment: the coordinated system had substantially more compute, while the practical definition of code sharing remained unclear. That does not negate Anthropic’s findings. It identifies the standard the next generation of evaluations must meet.

The future of agent teams will not be decided by the largest swarm. It will be decided by organizations that understand where autonomy should end, where a contract should begin and when adding another capable agent makes the whole system less capable.