Creating Multi-Agent Debate Systems: Architecture, Topologies, and Production Implementation

Multi-agent debate systems improve LLM accuracy by orchestrating collaborative critique sessions where independent agents iteratively refine answers until they converge on a consensus, using configurable communication topologies to balance information exchange against computational cost.

Creating multi-agent debate systems transforms single-model inference into an ensemble reasoning process, enabling autonomous agents to catch factual errors and challenge assumptions through structured argumentation. This guide examines the complete reference implementation in the rohitg00/ai-engineering-from-scratch repository (Phase 14, Lesson 25), detailing the Debater class architecture, communication topologies, and integration patterns required to deploy debate-based reasoning in production environments.

Core Architecture Components

The debate framework consists of three primary abstractions that separate agent logic from communication patterns and orchestration.

The Debater Agent Contract

At phases/14-agent-engineering/25-multi-agent-debate/code/main.py (lines 14-18), the Debater class is defined as a Python @dataclass containing a drift callable. This drift method implements the agent's reasoning update logic, accepting a question and list of peer answers to produce a revised response:

@dataclass
class Debater:
    name: str
    drift: Callable[[str, List[str]], str]  # question, peer_answers -> new_answer

Communication Topologies

The framework implements two distinct information-flow patterns in the same source file. The full_mesh_round function (lines 32-41) enables complete peer visibility where every agent observes all other proposals, while sparse_star_round (lines 43-57) implements a hub-and-spoke pattern that limits visibility to reduce token costs.

Orchestration and Convergence Tracking

The run_debate function (lines 60-82) serves as the central orchestrator. It accepts a list of Debater instances, a question string, the number of rounds, and a topology selector. The function tracks ops (operations) as a proxy for token expenditure and detects convergence when all agents return identical answers.

Information Flow Topologies: Full-Mesh vs. Sparse-Star

Choosing the right communication topology directly impacts both accuracy and computational cost.

Full-Mesh Topology

In full-mesh debate, every agent critiques every other agent's proposal each round. This maximizes information exchange and often accelerates convergence on factual consensus, but incurs N·(N‑1)·R total critique operations, where N is the agent count and R is the round count.

Sparse-Star Topology

The sparse-star topology designates one hub agent that observes all spoke proposals, while spoke agents only see the hub's answer. This architecture reduces operations to N·R + (N‑1) while preserving accuracy for many reasoning tasks, making it suitable for cost-sensitive applications.

Implementing a Multi-Agent Debate System

Running the Built-In Demo

Execute the reference implementation to observe convergence behavior on factual questions:

python3 phases/14-agent-engineering/25-multi-agent-debate/code/main.py

The script outputs convergence metrics including the final answer, the round where consensus first occurred, and total operations (ops):

--- capital_of_portugal (truth: Lisbon) ---
full_mesh    answer=Lisbon    converged_round=1    ops=6    CORRECT
sparse_star  answer=Lisbon    converged_round=1    ops=4    CORRECT

Customizing Agent Behavior

You can instantiate debaters with specific biases or correction dictionaries using the _make_debater helper, then invoke run_debate with your preferred topology:

from phases_14_agent_engineering_25_multi_agent_debate.code.main import (
    Debater, _make_debater, run_debate
)

# Create debaters with different knowledge biases

debaters = [
    _make_debater("alpha", bias="Paris", corrections={"capital_of_france": "Paris"}),
    _make_debater("beta", bias="Berlin", corrections={"capital_of_germany": "Berlin"}),
    _make_debater("gamma", bias="Rome", corrections={"capital_of_italy": "Rome"}),
]

answer, converged, ops = run_debate(
    debaters,
    question="capital_of_france",
    rounds=3,
    topology="sparse_star",   # Alternative: "full_mesh"

)

print(f"Final answer: {answer}, converged in round {converged}, ops={ops}")

Integrating Production LLM Endpoints

Replace the scripted drift function with calls to OpenAI, Anthropic, or other providers while maintaining the same contract:

import openai
from typing import List

def llm_drift(question: str, peer_answers: List[str]) -> str:
    prompt = f"""Question: {question}
Peer answers: {', '.join(peer_answers) or 'none'}

Provide a concise answer, improving on the peers if needed."""
    
    resp = openai.ChatCompletion.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.choices[0].message.content.strip()

# Instantiate real LLM debater

real_debater = Debater(name="gpt4", drift=llm_drift)

answer, conv, ops = run_debate(
    [real_debater, real_debater],
    question="Explain quantum entanglement",
    rounds=2,
    topology="full_mesh",
)

Security Note: When integrating production LLMs, wrap calls in verification gates using the sandbox pattern from phases/19-capstone-projects/25-verification-gates-observation-budget to control token budgets and prevent unsafe executions.

Integration Patterns for Production Systems

Orchestrator-Workers Pattern

Map each Debater instance to a separate worker process using Anthropic's orchestrator-workers pattern (Lesson 12), with a central dispatcher routing peer answers according to the selected topology.

LangGraph State Machines

The run_debate orchestration logic translates directly to LangGraph nodes, where each debate round represents a state transition. The hub agent in sparse-star topology becomes a router node, while spoke agents execute as child nodes with filtered state visibility.

OpenAI Agents SDK

Debate agents can be implemented as OpenAI Agents that invoke sparse_star_round or full_mesh_round as tool calls, enabling the debate framework to function within the OpenAI Agents SDK ecosystem while maintaining the same convergence tracking.

When to Use Multi-Agent Debate

Benefits:

• Factuality: Independent proposals expose contradictory facts, and cross-critiques force agreement on verifiable information.
• Rule Adherence: In structured domains like chess or code review, peers catch illegal moves or syntax violations that a single agent might miss.
• Bias Reduction: Multiple perspectives mitigate systematic errors inherent to any single model's training distribution.

Limitations:

• Latency: Each round executes serially, so N × R API calls multiply wall-clock time proportionally.
• Cost: Token usage grows linearly with agent count and rounds, making debate expensive for simple lookup tasks where single-model inference suffices.
• Convergence Failure: Highly subjective questions may never reach consensus, requiring round limits and fallback logic.

Summary

• The Debater class in main.py (lines 14-18) provides a minimal contract (name and drift callable) that supports both scripted and LLM-backed agents.
• Two topologies are provided: full-mesh (lines 32-41) for maximum information exchange at N·(N‑1)·R cost, and sparse-star (lines 43-57) for hub-spoke efficiency at N·R cost.
• The run_debate function (lines 60-82) handles orchestration, convergence detection, and operation counting.
• Production integration involves replacing the drift callable with LLM API calls while maintaining the question + peer_answers -> answer signature.
• Reusable scaffolding is available at phases/14-agent-engineering/25-multi-agent-debate/outputs/skill-debate.md for dropping debate capabilities into existing projects.

Frequently Asked Questions

What makes multi-agent debate more accurate than single-model reasoning?

Multi-agent debate systems improve accuracy by exposing factual contradictions through independent proposal generation. When agents with different initial answers critique each other, they must justify their positions using verifiable facts, effectively implementing a self-consistency check that single-model chain-of-thought prompting cannot replicate. This is particularly effective for domains requiring rule verification or factual consensus.

How do I choose between full-mesh and sparse-star topologies for my use case?

Select full-mesh when factual accuracy is critical and you can tolerate N² communication costs, such as in medical or legal reasoning where every agent must validate every claim. Choose sparse-star when operating under token budgets or latency constraints, as the hub agent effectively summarizes spoke perspectives, reducing operations to linear complexity while preserving the benefit of multi-perspective critique.

Can I combine different types of agents in a single debate?

Yes. The Debater dataclass abstraction allows mixing scripted agents (with hardcoded drift functions) and real LLM agents in the same debate array. This hybrid approach lets you use inexpensive scripted agents to check specific rules (like chess legality) while reserving costly LLM calls for open-ended reasoning tasks, optimizing the cost-accuracy tradeoff.

What happens if the agents fail to converge within the specified rounds?

The run_debate function returns the final answers, the convergence round (or None if unconverged), and the total operation count. You should implement fallback logic in your application to handle unconverged debates—typically by selecting the majority answer, triggering a tie-breaking agent, or escalating to human review when consensus is not reached within the budgeted rounds.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}