# Deer-Flow Architecture Overview: Inside ByteDance's LangGraph AI Super-Agent

> Explore the Deer-Flow architecture, ByteDance's AI super-agent. Learn how LangGraph and LangChain enable sandboxed execution, persistent memory, and tool integration via an 11-stage middleware pipeline.

- Repository: [Bytedance Inc./deer-flow](https://github.com/bytedance/deer-flow)
- Tags: architecture
- Published: 2026-03-08

---

**Deer-Flow is a full-stack AI super-agent harness built on LangGraph and LangChain that separates concerns into four runtime services—the Nginx reverse proxy, LangGraph Server, Gateway API, and embedded client—providing sandboxed execution, persistent memory, sub-agent delegation, and extensible tool integration through an 11-stage middleware pipeline.**

Deer-Flow is ByteDance's open-source AI agent framework designed for complex task execution with enterprise-grade isolation and extensibility. This architecture overview examines how the repository `bytedance/deer-flow` orchestrates LangGraph's stateful graphs with custom middleware chains, sandboxed environments, and a modular tool ecosystem to create a production-ready super-agent capable of handling multi-turn conversations, file operations, and concurrent sub-agent delegation.

## High-Level Service Layout

The Deer-Flow architecture deploys as a containerized stack with four distinct runtime layers unified behind a single reverse proxy. According to the backend README, Nginx listens on **port 2026** and routes traffic based on path prefixes.

### Reverse Proxy and Traffic Routing

Nginx terminates all external connections and implements path-based routing:

- `/api/langgraph/*` → LangGraph Server (port 2024)
- `/api/*` (all other routes) → Gateway API (port 8001)
- `/` → Next.js frontend

This design isolates the agent runtime from the REST façade, allowing the **LangGraph Server** to handle stateful graph execution while the **Gateway API** manages models, skills, memory, and file uploads through FastAPI endpoints.

### LangGraph Server vs. Gateway API

The **LangGraph Server** hosts the core agent runtime including the Lead Agent, middleware chain, and tool execution. It maintains conversation state via LangGraph's checkpointer system.

The **Gateway API** ([`backend/src/gateway/app.py`](https://github.com/bytedance/deer-flow/blob/main/backend/src/gateway/app.py)) exposes REST endpoints for configuration management, file uploads, and memory queries. Both services read from the same [`config.yaml`](https://github.com/bytedance/deer-flow/blob/main/config.yaml) and [`extensions_config.json`](https://github.com/bytedance/deer-flow/blob/main/extensions_config.json) files, supporting hot-reload of models and tools without restarting the stack.

### IM Channel Integration (Optional)

Deer-Flow optionally bridges external messaging platforms (Telegram, Slack, Feishu/Lark) through a lightweight **pub/sub hub** ([`message_bus.py`](https://github.com/bytedance/deer-flow/blob/main/message_bus.py)). These channels use the same LangGraph SDK as the frontend, connecting directly to the LangGraph Server to ensure consistent thread management across all interfaces.

## Lead Agent Runtime

At the heart of Deer-Flow is the **Lead Agent**, a configurable LangGraph agent instantiated through `make_lead_agent(config)` in [`src/agents/lead_agent/agent.py`](https://github.com/bytedance/deer-flow/blob/main/src/agents/lead_agent/agent.py). This factory function assembles the complete processing pipeline for every conversation turn.

### Dynamic Model Resolution

The agent selects LLM implementations via `create_chat_model`, which inspects the request or configuration to instantiate the appropriate provider (OpenAI, Anthropic, or local models). The model's capabilities—such as `supports_thinking` or `supports_vision`—dictate which middlewares activate during the run.

### System Prompt Construction

The `apply_prompt_template` function generates the system prompt by injecting:

- Enabled **skills** from the skills directory
- Retrieved **memory** context via `<memory>` tags
- **Sub-agent** instructions when delegation is enabled
- Sandbox environment variables

This occurs in [`src/agents/lead_agent/prompt.py`](https://github.com/bytedance/deer-flow/blob/main/src/agents/lead_agent/prompt.py), ensuring the model receives fully contextualized instructions before each inference.

## The Middleware Chain

Deer-Flow implements an **11-stage middleware pipeline** that executes in strict order for every turn. Defined in [`src/agents/lead_agent/agent.py`](https://github.com/bytedance/deer-flow/blob/main/src/agents/lead_agent/agent.py) via `_build_middlewares`, these components handle cross-cutting concerns from workspace isolation to memory extraction.

### Per-Turn Processing Pipeline

The middlewares execute sequentially:

1. **ThreadDataMiddleware** — Creates isolated per-thread directories (`workspace`, `uploads`, `outputs`) for file operations
2. **UploadsMiddleware** — Injects newly uploaded files into the conversation context
3. **SandboxMiddleware** — Acquires a sandbox (local or Docker) and stores its identifier in state
4. **DanglingToolCallMiddleware** — Patches missing `ToolMessage` objects when tool calls interrupt
5. **SummarizationMiddleware** — Reduces context window when token limits approach (optional)
6. **TodoListMiddleware** — Provides the `write_todos` tool for multi-step task tracking (plan mode only)
7. **TitleMiddleware** — Auto-generates conversation titles after the first exchange
8. **MemoryMiddleware** — Queues conversational snippets for asynchronous long-term memory extraction
9. **ViewImageMiddleware** — Injects base64 image data for vision models
10. **SubagentLimitMiddleware** — Enforces the maximum concurrent sub-agent count
11. **ClarificationMiddleware** — Intercepts `ask_clarification` tool calls and can interrupt execution

### Critical Ordering Constraints

Middleware ordering is architecturally significant. For example, `ViewImageMiddleware` must execute before `ClarificationMiddleware` to ensure image data is available to the model before any clarification logic processes. Similarly, `SandboxMiddleware` must acquire execution environments before tools attempt file operations.

## Sandboxed Execution Environment

The **Sandbox System** (`src/sandbox/`) provides isolated execution contexts for code and command operations, preventing the agent from accessing host resources directly.

### Abstract Interface and Providers

The [`sandbox.py`](https://github.com/bytedance/deer-flow/blob/main/sandbox.py) module defines an abstract interface with methods like `execute_command`, `read_file`, `write_file`, and `list_dir`. Two providers implement this interface:

- **LocalSandboxProvider** — Runs commands directly on the host filesystem (development mode)
- **AioSandboxProvider** (`src/community/`) — Executes commands inside isolated Docker containers or Kubernetes pods via the AIO provisioner

### Virtual Path Mapping

The sandbox layer implements virtual-to-physical path translation, presenting the agent with a clean namespace:

- `/mnt/user-data/workspace`, `/uploads`, `/outputs` → Physical per-thread directories created by ThreadDataMiddleware
- `/mnt/skills` → Read-only mount of the repository's `skills/` directory

This abstraction allows tools to use consistent paths regardless of the underlying provider.

## Sub-Agent Delegation System

Deer-Flow supports hierarchical agent delegation through the **Sub-Agent System** (`src/subagents/`), enabling the Lead Agent to spawn specialized workers for parallel task execution.

### Concurrency Limits and Timeouts

The `SubagentExecutor` manages execution using dual thread pools (scheduler and runner) with strict resource limits:

- **Maximum 3 concurrent sub-agents** (`MAX_CONCURRENT_SUBAGENTS = 3`)
- **15-minute timeout** per sub-agent task

Built-in agent types include `general-purpose` (full tool access) and `bash` (command execution specialist).

### Event Streaming Architecture

Sub-agents communicate state changes back to the Lead Agent via Server-Sent Events (SSE). Events include `task_started`, `task_running`, `task_completed`, and error states, enabling real-time monitoring of distributed work.

## Tool Ecosystem Integration

The Lead Agent aggregates capabilities from four distinct sources, resolved through `get_available_tools` in [`src/tools/__init__.py`](https://github.com/bytedance/deer-flow/blob/main/src/tools/__init__.py).

### Four-Source Tool Aggregation

1. **Configured tools** — Defined in [`config.yaml`](https://github.com/bytedance/deer-flow/blob/main/config.yaml) and resolved via Python reflection
2. **MCP tools** — Dynamically loaded from Model Context Protocol servers defined in [`extensions_config.json`](https://github.com/bytedance/deer-flow/blob/main/extensions_config.json)
3. **Built-in tools** — Core utilities including `present_files`, `ask_clarification`, `view_image`, and sandbox file operations (`bash`, `ls`, `read_file`, `write_file`, `str_replace`)
4. **Community tools** — Web search (Tavily), web fetch (Jina AI/Firecrawl), image search, and Docker sandbox providers

### Hot-Reloadable Configuration

Both [`config.yaml`](https://github.com/bytedance/deer-flow/blob/main/config.yaml) and [`extensions_config.json`](https://github.com/bytedance/deer-flow/blob/main/extensions_config.json) support hot reloading. The Gateway API watches these files to update available models, MCP server connections, and enabled skills without service restarts.

## Memory and Persistence

The **Memory Subsystem** (`src/agents/memory/`) provides long-term conversational context across threads.

### Asynchronous Extraction

The **Updater** component extracts user context, facts, and preferences from conversations using a dedicated LLM. The **Queue** debounces these updates (default **30 seconds**) and batches changes per thread to minimize API calls.

Data persists atomically in [`backend/.deer-flow/memory.json`](https://github.com/bytedance/deer-flow/blob/main/backend/.deer-flow/memory.json), ensuring durability across restarts.

### Runtime Injection

At inference time, the **MemoryMiddleware** retrieves the top-N most relevant facts and injects them into the system prompt within `<memory>` XML tags, allowing the model to reference historical preferences without exhaustive context windows.

## Developer Integration Patterns

Deer-Flow exposes two primary integration interfaces: the HTTP REST API and the **Embedded Python Client** ([`src/client.py`](https://github.com/bytedance/deer-flow/blob/main/src/client.py)).

### Single-Process Execution

The `DeerFlowClient` class provides a **single-process** alternative to the full HTTP stack, mirroring every dictionary-returning Gateway endpoint including `list_models`, `list_skills`, `upload_files`, and `get_memory`.

```python
from src.client import DeerFlowClient

client = DeerFlowClient()  # Reads config.yaml automatically

response = client.chat(
    "Analyze the attached CSV and generate a summary report.",
    thread_id="analysis-demo"
)

```

### Streaming and File Operations

The client supports LangGraph-compatible event streaming and file uploads with automatic format conversion:

```python

# Streaming events

for event in client.stream("Generate a project plan", thread_id="plan-123"):
    if event.type == "messages-tuple" and event.data["type"] == "ai":
        print(event.data["content"])

# File uploads with auto-conversion

result = client.upload_files(
    thread_id="report-gen",
    files=["./paper.pdf", "./data.xlsx"]
)

```

### Plan Mode Activation

Enable the **TodoListMiddleware** at runtime to activate structured task planning:

```python
client = DeerFlowClient(plan_mode=True)
client.chat("Create a data pipeline architecture", thread_id="pipeline-design")

```

## Summary

- Deer-Flow separates concerns into **Nginx** (proxy), **LangGraph Server** (agent runtime), **Gateway API** (REST façade), and an optional **Embedded Client** for library integration.
- The **Lead Agent** processes every turn through an **11-stage middleware chain** that handles sandboxing, file uploads, memory extraction, and sub-agent limits in strict sequence.
- **Sandboxed execution** supports both local and Docker/Kubernetes providers via virtual path mapping to per-thread directories.
- **Sub-agent delegation** enforces hard limits of **3 concurrent tasks** with **15-minute timeouts**, managed by `SubagentExecutor`.
- The **tool ecosystem** aggregates capabilities from YAML configuration, MCP servers, built-ins, and community extensions, all hot-reloadable.
- **Long-term memory** extracts facts asynchronously with 30-second debouncing and injects context into system prompts via `<memory>` tags.

## Frequently Asked Questions

### How does Deer-Flow handle concurrent sub-agent execution?

Deer-Flow enforces a hard concurrency limit of **three parallel sub-agents** (`MAX_CONCURRENT_SUBAGENTS = 3`) through the `SubagentLimitMiddleware`. The `SubagentExecutor` manages these workers using dual thread pools for scheduling and execution, with each sub-agent subject to a **15-minute timeout** before forced termination.

### What is the purpose of the middleware chain in Deer-Flow?

The middleware chain in [`src/agents/lead_agent/agent.py`](https://github.com/bytedance/deer-flow/blob/main/src/agents/lead_agent/agent.py) processes every conversation turn through 11 sequential stages that handle cross-cutting concerns including workspace isolation (`ThreadDataMiddleware`), sandbox acquisition (`SandboxMiddleware`), memory extraction (`MemoryMiddleware`), and vision model preparation (`ViewImageMiddleware`). The strict ordering ensures dependencies are satisfied—for example, image data injection must occur before clarification handling.

### How does the sandbox system isolate agent file operations?

The **Sandbox System** (`src/sandbox/`) provides an abstraction layer with two providers: `LocalSandboxProvider` for host filesystem access and `AioSandboxProvider` for Docker/Kubernetes isolation. It maps virtual paths (`/mnt/user-data/workspace`, `/uploads`, `/outputs`) to physical per-thread directories created by `ThreadDataMiddleware`, ensuring agents cannot access files outside their allocated workspace regardless of the execution backend.

### Can Deer-Flow operate without the HTTP services?

Yes. The **Embedded Client** ([`src/client.py`](https://github.com/bytedance/deer-flow/blob/main/src/client.py)) provides a single-process execution mode that instantiates the full Deer-Flow stack—including the Lead Agent, middleware chain, and tools—without launching Nginx or the Gateway API. This mode supports all core features including streaming events, file uploads, and memory persistence, making it suitable for library integration and automated testing.