Multi-Agent Systems

A single agent (Chapter 38) has one viewpoint and one set of blind spots. Putting several agents on the same task promises to cover those blind spots, and the default way to cash that promise is to vote. But multi-agent reliability is not a simple law of large numbers. Voting among similar agents buys less than the math suggests; structured disagreement buys more; and the system has to separate liveness failures from safety failures so it escalates recoverable failures and blocks silent ones.

The default move: more agents, take the consensus

The dominant pattern for making a multi-agent system reliable is to run more agents and take the consensus: vote across $N$ samples, ensemble across model families, add a judge. The implicit assumption is that agreement is evidence and disagreement is noise to be averaged away. That intuition came from places where it held, statistical estimation, classical fault-tolerant computing, and machine-learning ensembles trained on independently sampled data. None of those assumptions survive contact with stochastic agents driven by large language models. Treating agreement as a proxy for correctness has three load-bearing problems, and the rest of this section takes them one at a time.

Failures are correlated

Classical Byzantine fault tolerance does not require probabilistic independence in its theorem; it requires a credible bound on how many replicas can be faulty at once. The $f < N/3$ threshold is meaningful only when fewer than a third of replicas can share the same Byzantine failure mode. Two agents drawing from the same base model, the same training distribution, and often the same prompt template do not have independent failure modes. They co-hallucinate (a hallucination is a confident but wrong output; to co-hallucinate is for several agents to land on the same wrong output together). They share the same blind spots, the same bias toward fluent-sounding wrong answers, the same weakness on the same inputs. Berdoz et al. measured this directly: valid scalar consensus among LLM agents drops from 46.6% at $N=4$ to 33.3% at $N=16$ even without any adversary (Berdoz et al. 2026). Adding agents made it worse, not better.

Here $N$ is the replica count and $f$ is the number allowed to be Byzantine. The threshold is not magic redundancy; it is a statement about fault diversity. If many replicas can fail for the same reason, the inequality has already lost the assumption that made it useful.

Agreement is cheap to manufacture

A small amount of conformity pressure, a leading prompt, or a shared in-context example can drive agents into convergent but incorrect answers. The Free-MAD work gives a modern version: in consensus-driven debate, agents that initially produce correct responses can be pulled toward incorrect ones, and final majority voting can degrade reasoning performance (Cui et al. 2025). When the system is graded on agreement, agents learn to agree, even when the right move is to flag that they cannot.

Self-evaluation is biased

Panickssery et al. showed LLM judges systematically favor their own generations (Panickssery et al. 2024), and position bias in pairwise judges is well documented (Shi et al. 2025) (Chapter 50). An ensemble of judges drawn from the same family inherits the same biases. A verification layer that is a cheaper echo of the layer it verifies cannot tell you anything you did not already know. The accurate reading is that agreement among similar models is weak evidence of correctness. It is sometimes evidence of nothing more than that the input was unambiguous enough for any plausible decoder to land on the same answer.

Disagreement as signal, not noise

The deeper move is to stop treating disagreement as noise. In an idealized debate, a single competent honest critic is sufficient to expose a flaw, no matter how many other critics are lazy or wrong (Irving et al. 2018). This asymmetry is the core design idea, and it inverts the orchestration. Instead of symmetric voters whose verdicts are pooled, the roles are specialized: a proposer produces an artifact, independent critics each attack one distinct axis of it, and a cheap judge adjudicates only the disputed claim.

A small formal contrast makes the asymmetry precise. The plain version is that independent errors tend to cancel, while correlated errors compound: when critics fail for unrelated reasons their misses multiply away to almost nothing, but when they share a failure mode their misses pile up and extra critics add little. The math below states exactly that. Let $p_i \in [0, 1]$ be the probability that critic $i$ catches a flaw on the axis it evaluates, and let $\rho \in [0, 1]$ be the pairwise correlation between verdicts.

Aggregational verification with $k$ voters under majority vote degrades as correlation rises. As $\rho \to 1$ the probability a flaw survives the ensemble approaches the probability that the single shared failure mode misses it, and adding voters does not help:

$$\Pr[\text{flaw survives} \mid \rho \to 1] \;\to\; \Pr[\text{shared mode misses flaw}]$$

Adversarial verification with $k$ independent critics on distinct axes, where one caught flaw suffices, factors over critics. Under independence this is exact; any residual positive correlation only raises survival above the product, so the product is the best case you are buying:

$$\Pr[\text{flaw survives}] \;=\; \prod_{i=1}^{k} (1 - p_i)$$

Here each $p_i$ is the chance that critic $i$ catches the flaw on its assigned axis. The product only buys something if the $p_i$ cover different ways the artifact can be wrong. Write $p_i = c_i d_i$, where $c_i$ is the probability critic $i$ selects the relevant attack surface and $d_i$ is the probability it turns that surface into a concrete objection. Duplicating critics on the same surface mostly raises $d_i$ locally; forcing topic diversity raises coverage, roughly $1 - \prod_i (1 - c_i)$. A single competent honest critic on the relevant axis, $p_i = 1$, collapses the product to zero. This is the asymmetry that lets adversarial verification dominate consensus when failures are correlated and at least one independent honest critic exists.

Try sliding the correlation up. In the shared-latent mixture $\Pr[\text{survives}] = \rho(1-p) + (1-\rho)(1-p)^k$, $p$ is a single critic's chance of catching the flaw, $k$ is the panel size, and $\rho$ is the share of failure explained by a common latent mode. The formula says why voting flattens as $\rho$ rises while the independent product keeps driving survival down.

import numpy as np
import matplotlib.pyplot as plt

p = 0.5                       # per-agent chance of catching the flaw
k = np.arange(1, 11)          # number of agents / critics
for rho in [0.0, 0.3, 0.6, 0.9]:
    survives = rho * (1 - p) + (1 - rho) * (1 - p) ** k
    plt.plot(k, survives, marker="o", label=f"voting, rho={rho}")
plt.plot(k, (1 - p) ** k, "k--", lw=2, label="adversarial (rho=0 product)")
print("rho=0.9 survival at k=10:", round(rho := 0.9, 2),
      "->", round(0.9 * (1 - p) + 0.1 * (1 - p) ** 10, 4))
print("adversarial survival at k=10:", round((1 - p) ** 10, 4))
plt.xlabel("agents (k)"); plt.ylabel("Pr[flaw survives]")
plt.legend(); plt.title("Adding voters buys little when failures correlate")
plt.show()

Disagreement also obeys a monotonicity invariant that voting violates. Here $U_t$ is the set of unresolved attacks after round $t$, $R_t \subseteq U_t$ is the subset resolved by a fix, concession, or convincing defense, and $A_{t+1}$ is the set of new attacks introduced next round:

$$U_{t+1} = (U_t \setminus R_t) \cup A_{t+1}$$

Here $U_t \setminus R_t$ removes only attacks that were actually answered, and the union with $A_{t+1}$ adds fresh attacks discovered in the next round. The formula is the bookkeeping rule behind adversarial review: unresolved objections persist until resolved. No unresolved objection disappears because more agents voted against it; it disappears only by being resolved. Majority aggregation violates this by design, since a real minority objection can be buried under a larger pile of shallow agreement. The implication for design is precise: you do not need every critic to be competent, you need a process that surfaces the competent objection when it exists and terminates on the contested points rather than averaging them away. This is closer to proof checking, property testing, and red-team review than to ensemble prediction. The verifier does not reproduce the whole artifact; it forces a disputed claim into a witness small enough for a cheap judge to inspect: a failing test, a vulnerable input, a violated invariant, a missing citation. Figure 43.2 contrasts the two orchestrations.

Pooling and structuring both arrange agents that see the same task. A third arrangement chains them, each agent consuming the previous one's output, which is how most multi-step pipelines are actually built. Its characteristic failure is the mirror of consensus's: not correlated agreement but cascading error, where one stage's hallucination becomes the next stage's premise and compounds down the chain, the cross-agent form of the within-trajectory compounding that Chapter 52 grades. The same agents-verification probe runs these sequential chains alongside the parallel panels, seeding an early error to measure how far it travels and whether a later stage ever catches it.

The fourth arrangement, and the one production converged on, is delegation: agents that see different slices of the task rather than the same one. An orchestrator decomposes the task and spawns ephemeral subagents, each working its slice in its own context window and returning a compressed summary rather than a transcript. The isolation is the point: parallel subagents explore without contaminating each other's context, and the lead agent reasons over summaries instead of raw tool output. Anthropic's research system is the measured exemplar. A lead Claude Opus 4 agent coordinating parallel Sonnet 4 subagents outperformed single-agent Opus 4 by 90.2% on an internal research evaluation, at roughly fifteen times the tokens of a chat (Anthropic 2025); the same shape ships as subagents in Claude Code and handoffs in OpenAI's Agents SDK. Its characteristic failure is drift: a subagent whose objective, output format, or boundaries are underspecified wanders, and the lead agent cannot see the wandering because isolation cut the channel that would have shown it. Chapter 41 measured the sandbox side of this isolation; here it is the coordination structure itself.

Two failures that look alike

The lineage runs from pooling to structure. The starting point is the ensemble: independent samples, averaged or voted, which works when the samples are genuinely independent. Byzantine fault tolerance is the distributed-systems refinement of that idea, and reading it carefully is what reveals why the naive translation to agents fails. Two of its contributions survive translation, and they are the two practitioners tend to skip.

The first is the separation of liveness from safety. A system can fail by producing the wrong answer, a safety violation, or by failing to produce any answer at all, a liveness violation. The empirical results above are striking precisely because LLM consensus failures are mostly liveness failures: the agents do not converge on a wrong answer, they fail to converge at all (Berdoz et al. 2026). A liveness failure is recoverable, escalated to a human or a different model. Silent, a safety failure ships. A system that collapses both into one accuracy number optimizes the wrong axis. The second is the recognition that correlated failure is how a fault bound becomes false in practice. The fix is not more replicas but breaking the correlation: heterogeneous models, prompts, tooling, and where possible heterogeneous formulation of the question itself.

This split is the chapter's central routing decision, and Figure 43.3 draws it: a single accuracy number collapses two failures that demand opposite responses.

From debate to a bounded judge

From the liveness-safety split the line continues into debate. Irving, Christiano and Amodei framed AI safety as a debate judged by a weaker verifier (Irving et al. 2018), and Brown-Cohen, Irving and Piliouras gave the formal extension: soundness holds under a large compute asymmetry, where the honest strategy uses polynomial simulation steps while the dishonest strategy may use exponentially many (Brown-Cohen et al. 2023). The complexity-theoretic intuition is that under optimal play debate can answer PSPACE questions (Irving et al. 2018), because a judge inspecting only the single disputed leaf where the players' disagreement bottoms out can verify outcomes it could never have produced unaided (Brown-Cohen et al. 2023). That is the modern destination of the arc: specialized adversarial roles with a bounded judge, not a wider pool of symmetric voters.

What it costs

Structured disagreement is not free, and the boundaries are where the source material leaves the unsettled parts visible.

• Heterogeneity has a cost worth paying. Mixing critic models, for example Codex critics against a Claude proposer, introduces integration complexity, provider drift, and a coordination tax. It is also the only thing that restores any useful version of the fault diversity the verification math requires. There is no known shortcut.
• Cardinality is unknown. One critic per topic is the floor; the ceiling is unclear. More critics on the same topic should give diminishing returns under correlated failure, but the shape of that curve across model families has no clean empirical story yet.
• Protocols drift. A debate that runs too long drifts off the original claim, and contention scores can be inflated by stylistic friction rather than substantive disagreement. The right termination criterion is unsettled; steady-state detection plus a hard round cap catches common cases but is not optimal.
• Liveness escalates, safety blocks. A system that distinguishes "the agents could not converge" from "the agents converged on the wrong answer" can route the first to a human and refuse to ship the second. One that cannot tell them apart does the wrong thing on both. This is the payoff of keeping the two axes separate rather than collapsing them into a single score, and it ties directly into oversight (Chapter 55) and agent evaluation (Chapter 52).

Gating an action on agreement

So far the panel produces an answer. When that answer triggers a side effect, a refund issued, a branch merged, a production change shipped, the question shifts from which answer is best to whether the system should act at all, and consensus becomes an authorization gate rather than a selection rule. The liveness-safety split maps onto it directly: require a quorum to act, and escalate a failure to converge to a human instead of guessing. The threshold is the dial. A one-of-$N$ gate acts if any agent approves, which maximizes liveness and minimizes safety; a unanimous gate acts only if all agree, which does the reverse; the $N - \lfloor N/3 \rfloor$ supermajority is the classical Byzantine quorum, chosen so that no minority of faulty participants can force or block the decision. Each setting trades a false action against a false block, and the right point depends on the cost of acting wrongly versus the cost of stalling.

The gate matters most when a participant is not merely wrong but adversarial. A sub-agent reached by untrusted input can be prompt-injected (Chapter 58) into emitting a confident lie or a subtle misdirection, and at the output an injected lie and an honest co-hallucination look alike. They differ in the threat model. Heterogeneity breaks honest correlation, but an attacker who can inject one agent can often inject several through the same channel, so the $f < N/3$ bound holds only as far as the compromises are independent. A consensus gate is therefore defense in depth, not a guarantee: it raises the bar so that acting wrongly requires corrupting a quorum rather than a single agent, which is the same posture authorization takes in Chapter 56. The research probe agents-verification measures exactly this, sweeping the quorum threshold and injecting Byzantine agents to separate the liveness cost of a strict gate from the safety it buys.

Heterogeneous orchestration also raises a wiring question this chapter has so far assumed away: agents from different vendors have to speak something. The Agent2Agent protocol (A2A) is the emerging answer, an MCP-analogue at the agent-to-agent boundary. Each agent publishes an agent card, a JSON document describing its capabilities, endpoint, and authentication, and peers drive a task through a defined lifecycle over HTTP, JSON-RPC, and server-sent events. Google announced the protocol in April 2025, contributed it to the Linux Foundation that June, and it reached v1.0 in April 2026 with more than 150 organizations behind it (The Linux Foundation 2025; The Linux Foundation 2026). The same wire contract that makes a cross-vendor panel practical is also an attack surface: an endpoint that accepts tasks from other agents accepts injected ones, which is exactly the adversarial participant the quorum gate above is sized against.

The shipped protocol

The protocol that falls out is small. Run a proposer to produce an artifact. Fork independent critics with no shared context. Each critic picks a distinct attack topic in its first round, security, performance, internal consistency, evidence gap, and later critics are told which topics are taken so they pick something else. Run cross-examination per critic until the proposer concedes or the critic gives up. Apply any concessions, then surface only the unresolved disputes, ranked by how long each survived attack. Figure 43.4 shows the per-critic loop and how it terminates.

The judging layer should be cheaper and dumber than the proposing layer; this is the doubly-efficient debate intuition (Brown-Cohen et al. 2023). If the judge has to be as smart as the proposer, nothing was bought. A concrete realization keeps no model in the judge loop at all, scoring each attack by a pure contention count:

$$\text{score}(a) = \text{rounds\_survived}(a) + \mathbf{1}[\text{re\_attacked}(a)]$$

Here the indicator adds one point if an unresolved attack is rediscovered later, so survival and recurrence both raise the score. Two design properties make this work in practice. Claims need addresses: every nontrivial claim needs a handle, a file path, test case, source citation, requirement id, so critics have something concrete to attack and proposers something concrete to repair. Without addressability the review collapses into rhetoric. And the critic must reach the proposer through a verbatim channel, a plain user turn rather than a template that would distort the proposer's normal defense behavior. The shipped reference for this design is Latere's Adversarial Review capability, which forks a coding session via Claude Code's --fork-session, runs the cross-examination in the fork, and writes only the unresolved disputes to disk, leaving the root transcript byte-identical. The research probe behind it, agents-verification, measures liveness and safety on independent axes across parallel-consensus and sequential-chain experiments rather than collapsing them into one accuracy number.

Further reading