# How Multi-Agent Systems and Swarms Work in the AI Engineering Curriculum

> Discover how multi-agent systems and swarms break AI complexity. Explore architectural patterns like pipelines and swarms in this AI engineering curriculum.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-06-20

---

**Multi-agent systems break the single-agent ceiling by distributing complex tasks across specialized agents that communicate through shared state, using architectural patterns like pipelines, fan-out/fan-in, orchestrator-worker, and peer swarms.**

The AI Engineering from Scratch curriculum treats multi-agent systems (MAS) as essential infrastructure for production workloads that exceed the context limits and serial processing constraints of single LLM loops. According to the source materials in `rohitg00/ai-engineering-from-scratch`, these systems overcome context saturation and sequential bottlenecks by partitioning workloads across role-specific agents with focused system prompts. The curriculum provides minimalist, standard-library implementations that demonstrate how to design, prototype, and evaluate these architectures without external dependencies.

## Breaking the Single-Agent Ceiling

The curriculum identifies a critical limitation in single-agent architectures: the **single-agent ceiling**. When one LLM attempts to juggle research, coding, reviewing, and testing within the same prompt, the context window (approximately 200,000 tokens) quickly saturates, causing role confusion and serial bottlenecks. As documented in [`phases/16-multi-agent-and-swarms/01-why-multi-agent/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/01-why-multi-agent/docs/en.md), this saturation forces the model to context-switch between competing objectives, degrading output quality and increasing latency.

Multi-agent systems solve this by **enforcing role boundaries**. Each agent maintains a narrow system prompt and handles a specific concern, passing structured messages through a shared state rather than accumulating all context in a single window.

## Four Core Multi-Agent Patterns

The curriculum defines four distinct architectural patterns for coordinating multiple agents, each suited to different workflow constraints.

### Pipeline Pattern

The **Pipeline** pattern executes agents sequentially, where each agent's output becomes the next agent's input. This creates an assembly-line workflow (research → coder → reviewer) with deterministic ordering and clear data dependencies. The pattern is implemented in the curriculum's examples using a `StaticOrchestrator` that enforces a fixed execution order.

### Fan-Out / Fan-In Pattern

**Fan-out / fan-in** enables parallelization by spawning multiple agents simultaneously to process independent subtasks, then merging their results before the next stage begins. This pattern maximizes throughput when tasks can be partitioned horizontally, such as analyzing multiple documents or testing different code variations concurrently.

### Orchestrator-Worker Pattern

The **Orchestrator-Worker** pattern introduces a dedicated controller agent that decides which worker executes next and aggregates their responses. Unlike the static pipeline, the orchestrator dynamically selects the next speaker based on current state, enabling conditional workflows and error recovery loops. The curriculum implements this via the `LLMSelectorOrchestrator` class, which uses an LLM-driven selector to determine the next agent.

### Peer Swarm Pattern

**Peer Swarm** eliminates centralized control entirely. Multiple identical agents share a message pool or queue with no fixed leader; coordination emerges from local interactions as agents read the shared state and decide when to contribute. This decentralized approach, demonstrated in [`phases/16-multi-agent-and-swarms/09-parallel-swarm-networks/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/09-parallel-swarm-networks/code/main.py), enables robust, self-organizing behavior for search and optimization tasks.

## Architectural Primitives in Python

The curriculum provides a reference implementation in [`phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py) that builds multi-agent systems using only Python's standard library. This minimalist approach teaches the fundamental mechanics without obscuring them behind framework abstractions.

The implementation defines four core primitives:

- **Agent**: A named entity encapsulating a system prompt and a *policy* function that generates messages.
- **Handoff**: A message field indicating which agent should speak next, enabling dynamic routing.
- **SharedState**: A thread-safe message bus using `threading.Lock()` to store all conversation history.
- **Orchestrator**: The execution controller that determines agent order, with three concrete implementations:

```python
import threading

# Shared message bus -------------------------------------------------

class SharedState:
    messages: list[dict] = []
    _lock = threading.Lock()

    def append(self, msg):
        with self._lock:
            self.messages.append(msg)

    def snapshot(self):
        with self._lock:
            return list(self.messages)

# Simple agents ------------------------------------------------------

class Agent:
    def __init__(self, name, policy):
        self.name = name
        self.policy = policy

    def run(self, state):
        msg = self.policy(state)
        msg.setdefault("from", self.name)
        return msg

# Orchestrators -------------------------------------------------------

class StaticOrchestrator:
    """Run agents in a predetermined order."""
    def __init__(self, order): 
        self.order = order
    
    def run(self, team, state, max_steps=10):
        for name in self.order[:max_steps]:
            state.append(team[name].run(state))

class HandoffOrchestrator:
    """Current agent returns the name of the next agent."""
    def __init__(self, start): 
        self.start = start
    
    def run(self, team, state, max_steps=10):
        cur = self.start
        for _ in range(max_steps):
            if cur not in team: 
                return
            msg = team[cur].run(state)
            state.append(msg)
            nxt = msg.get("handoff", "done")
            if nxt == "done": 
                return
            cur = nxt

class LLMSelectorOrchestrator:
    """Select next speaker via a selector (LLM in production)."""
    def __init__(self, start, selector): 
        self.start, self.selector = start, selector
    
    def run(self, team, state, max_steps=10):
        cur = self.start
        for _ in range(max_steps):
            if cur not in team: 
                return
            msg = team[cur].run(state)
            state.append(msg)
            cur = self.selector(state, team)

```

Running `python -m phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py` demonstrates how **the same agents and shared state** produce different execution traces solely by swapping the orchestrator, illustrating the separation of concerns between agent logic and control flow.

## Multi-Agent Reinforcement Learning

Beyond LLM-based systems, the curriculum covers **Multi-Agent Reinforcement Learning (MARL)** in [`phases/09-reinforcement-learning/10-multi-agent-rl/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/09-reinforcement-learning/10-multi-agent-rl/docs/en.md). In this paradigm, agents are treated as part of the environment itself, requiring **centralized training with decentralized execution** (CTDE). 

The source material emphasizes that MARL agents need explicit communication protocols to share observations and coordinate actions, unlike the message-passing of LLM agents. This distinction matters when choosing between learning-based policies (which require reward signals) and prompt-based policies (which follow fixed system instructions).

## When Not to Use Multi-Agent Systems

The curriculum explicitly warns against over-engineering. According to [`phases/16-multi-agent-and-swarms/01-why-multi-agent/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/01-why-multi-agent/docs/en.md), you should avoid MAS when:

- The task fits within a single context window without truncation.
- The workflow does not require specialized prompts or domains.
- The coordination overhead (latency, token costs, failure modes) outweighs the benefits of parallelization.

The repository includes a decision prompt template at [`phases/16-multi-agent-and-swarms/01-why-multi-agent/outputs/prompt-multi-agent-decision.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/01-why-multi-agent/outputs/prompt-multi-agent-decision.md) to automate this evaluation.

## Summary

- **Multi-agent systems** overcome the single-agent ceiling by distributing context across specialized agents with bounded responsibilities.
- **Four architectural patterns** exist: Pipeline (sequential), Fan-out/Fan-in (parallel), Orchestrator-Worker (centralized dynamic), and Peer Swarm (decentralized emergent).
- The curriculum provides **stdlib-only implementations** of Agent, Handoff, SharedState, and three Orchestrator types (Static, Handoff, and LLMSelector) in [`phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py).
- **Multi-agent RL** requires centralized training with decentralized execution and specific communication protocols distinct from LLM message passing.
- Use the provided decision prompt to determine if the coordination costs of MAS are justified for your specific task.

## Frequently Asked Questions

### What is the "single-agent ceiling" in AI engineering?

The single-agent ceiling occurs when an LLM's context window (approximately 200,000 tokens) fills up trying to juggle multiple responsibilities like research, coding, reviewing, and testing simultaneously. This causes context saturation, role confusion, and sequential bottlenecks that degrade performance and increase latency.

### How does the Peer Swarm pattern differ from Orchestrator-Worker?

Peer Swarm uses a decentralized architecture where multiple identical agents share a message pool or queue without a fixed leader, allowing coordination to emerge from local interactions. In contrast, Orchestrator-Worker relies on a dedicated controller that actively decides which worker agent executes next and aggregates their outputs, providing deterministic control flow.

### Can I build multi-agent systems without external libraries like LangGraph or AutoGen?

Yes. The curriculum demonstrates a complete primitive model using only Python's standard library in [`phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/16-multi-agent-and-swarms/04-primitive-model/code/main.py). It implements thread-safe shared state with `threading.Lock`, agent policies, and three orchestrator variants (StaticOrchestrator, HandoffOrchestrator, and LLMSelectorOrchestrator) without requiring external dependencies.

### When should I choose multi-agent reinforcement learning over LLM-based agents?

Choose MARL when agents must learn policies through environmental interaction and reward signals, as detailed in [`phases/09-reinforcement-learning/10-multi-agent-rl/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/09-reinforcement-learning/10-multi-agent-rl/docs/en.md). Use LLM-based multi-agent systems for deterministic workflows where agents follow fixed policies defined by system prompts rather than learned behaviors, or when you need the coordination patterns (pipeline, fan-out) suited for language model capabilities.