What Is bytedance/deer-flow? The Open-Source Super-Agent Harness Explained

bytedance/deer-flow is an open-source super-agent harness that orchestrates LLM agents, sub-agents, long-term memory, sandboxed execution environments, and extensible skills through a configurable middleware pipeline.

DeerFlow provides a full-stack system developed by ByteDance for building autonomous research and automation platforms. It packages a LangGraph agent runtime, FastAPI gateway, and Next.js frontend into a cohesive architecture that can be deployed as a complete service or embedded directly into Python applications via the DeerFlowClient.

System Architecture Overview

The platform follows a modular service-oriented design defined in backend/docs/ARCHITECTURE.md. A Nginx reverse proxy (port 2026) routes traffic between the browser-based UI, the LangGraph runtime (port 2024), and the FastAPI gateway (port 8001).

The topology connects these components to shared configuration files (config.yaml and extensions_config.json) and sandbox providers that isolate code execution. This separation allows the lead agent to process requests while external tools and skills execute in controlled environments.

The Lead Agent and Middleware Chain

The entry point for every AI interaction is the lead agent defined in src/agents/lead_agent/agent.py. The make_lead_agent function constructs a middleware chain that preprocesses every request before invoking the LLM model.

The pipeline processes requests through ten distinct middleware stages:

• ThreadDataMiddleware – Initializes per-thread directories including workspace, uploads, and outputs.
• UploadsMiddleware – Injects file metadata into the conversation history.
• SandboxMiddleware – Acquires an isolated execution environment (Docker or local).
• SummarizationMiddleware – Reduces context volume when approaching token limits, configured via summarization_config.py.
• TodoListMiddleware – Enables plan mode for structured multi-step task management.
• TitleMiddleware – Auto-generates conversation titles after the first exchange.
• MemoryMiddleware – Queues conversations for long-term persistence.
• ViewImageMiddleware – Adds vision capabilities for multimodal models.
• ClarificationMiddleware – Handles explicit clarification requests from the model.
• SubagentLimitMiddleware – Enforces concurrency limits on sub-agent execution.

Model resolution follows a hierarchy: request override → custom agent config → global default (_resolve_model_name). The system automatically disables "thinking" modes for models that do not support them.

Tools, Skills, and Extensibility

DeerFlow exposes capabilities through two primary mechanisms:

Built-in tools (src/tools/tools.py) provide core primitives including web_search, web_fetch, bash, read_file, write_file, and view_image. These execute within the sandboxed environment to prevent system contamination.

MCP tools (Model Context Protocol) are defined in extensions_config.json and loaded dynamically via src/mcp/manager.py. This allows integration with external services like GitHub without modifying core code.

Skills are Markdown-driven modules stored in skills/public/*. Each skill contains a front-matter header defining its name, description, and allowed tools. The src/skills/loader.py parses these files and injects them into the system prompt as contextual capabilities.

Memory and Persistence

Long-term memory stores user and session facts in JSON format under .deer-flow/threads/{thread_id}. The MemoryMiddleware (src/agents/middlewares/memory_middleware.py) captures conversation snippets and queues them for persistence.

The MemoryUpdater (src/agents/memory/updater.py) handles debounced writes and retrieval operations, ensuring the agent retains context across sessions without blocking the main execution thread. Configuration for memory behavior resides in src/config/memory_config.py.

Sandboxed Execution Environment

DeerFlow implements two sandbox providers to isolate tool execution:

• LocalSandboxProvider (src/sandbox/local.py) executes commands directly on the host for development scenarios.
• AioSandboxProvider (src/community/...) provides Docker-based isolation for production deployments.

Both providers expose a virtual filesystem mapping /mnt/user-data/ to thread-specific storage paths (.deer-flow/threads/<id>/user-data/), ensuring file operations remain scoped to the active conversation.

Configuration Management

The system relies on two primary configuration files watched for hot-reloading:

• config.yaml – Defines available models, tool settings, sandbox modes, and summarization triggers (documented in backend/docs/CONFIGURATION.md).
• extensions_config.json – Toggles built-in skills and registers external MCP servers.

These files are parsed by modules in src/config/ (including app_config.py and model_config.py) and validated at runtime.

Usage Patterns

Embedded Python Client

For programmatic integration without HTTP overhead, DeerFlow ships src/client.py containing the DeerFlowClient class. This client lazily instantiates the lead agent and provides synchronous and streaming interfaces:

from src.client import DeerFlowClient

# Initialize with automatic config.yaml loading

client = DeerFlowClient()

# Simple chat

response = client.chat("Analyze the latest trends in renewable energy")
print(response)

# Streaming response

for event in client.stream("Explain quantum computing"):
    if event.type == "messages-tuple" and event.data["type"] == "ai":
        print(event.data["content"], end="", flush=True)

The client exposes methods mirroring the REST API: list_models(), list_skills(), upload_files(), list_uploads(), and get_artifact().

Full-Stack Deployment

Deploying the complete system requires Docker Compose (docker/docker-compose-dev.yaml), which orchestrates:

1. Nginx – Reverse proxy and static file server for the UI
2. LangGraph Server – Hosts the lead agent runtime
3. FastAPI Gateway (src/gateway/app.py) – Exposes REST endpoints for model management, file uploads, and artifact retrieval

The Next.js frontend communicates through Nginx to reach both the gateway and LangGraph endpoints, providing a complete web interface for non-technical users.

Practical Code Examples

Uploading Files and Referencing Them

client = DeerFlowClient()

# Upload to specific thread

upload_res = client.upload_files(
    thread_id="research-123", 
    files=["./annual_report.pdf"]
)

# Reference via virtual path

response = client.chat(
    "Summarize the uploaded PDF in three bullet points",
    thread_id="research-123"
)

# The agent sees the file at /mnt/user-data/uploads/annual_report.pdf

Enabling Plan Mode for Complex Tasks

# Initialize with plan mode enabled

client = DeerFlowClient(plan_mode=True)

complex_task = """You need to:
1. Search for recent Python 3.12 performance benchmarks
2. Create a comparison table
3. Save results to a markdown file"""

response = client.chat(complex_task)

# TodoListMiddleware automatically tracks sub-task completion

Working with MCP Tools

# List available models and skills

models = client.list_models()
skills = client.list_skills()

# Skills appear in the system prompt based on extensions_config.json settings

print(f"Available models: {models}")
print(f"Active skills: {skills}")

Summary

• bytedance/deer-flow is a comprehensive super-agent platform combining LangGraph orchestration, sandboxed execution, and persistent memory.
• The lead agent (src/agents/lead_agent/agent.py) processes requests through a ten-stage middleware pipeline handling everything from file uploads to context summarization.
• Configuration flows through config.yaml and extensions_config.json, supporting hot-reloading without service restarts.
• Two deployment modes exist: embedded Python client (src/client.py) for library usage, and full-stack Docker deployment with Nginx, FastAPI, and Next.js.
• Sandboxed execution supports both local development and Docker-isolated production environments via pluggable providers.

Frequently Asked Questions

What is the lead agent in DeerFlow?

The lead agent is the central LangGraph runtime entry point defined in src/agents/lead_agent/agent.py. It constructs a middleware chain that preprocesses requests—including thread initialization, sandbox acquisition, memory updates, and model resolution—before invoking the LLM. This architecture allows DeerFlow to handle complex multi-step workflows while maintaining clean separation between preparation logic and model inference.

How does DeerFlow handle long-term memory?

Long-term memory persists in JSON files stored under .deer-flow/threads/{thread_id}. The MemoryMiddleware captures conversation snippets during processing, while the MemoryUpdater (src/agents/memory/updater.py) manages asynchronous writes to disk. This system allows agents to recall facts about users and previous sessions across restarts, with configuration controlled via src/config/memory_config.py.

Can DeerFlow operate without the web interface?

Yes. DeerFlow provides the DeerFlowClient class in src/client.py for embedded Python usage. This client loads config.yaml directly and exposes methods like chat() and stream() that mirror the external REST API, enabling integration into existing Python applications, scripts, or Jupyter notebooks without running the Nginx or Next.js services.

What execution environments does DeerFlow support for tools?

DeerFlow implements two sandbox providers: LocalSandboxProvider (src/sandbox/local.py) for direct host execution during development, and AioSandboxProvider (located in src/community/) for Docker-based isolation in production. Both expose a virtual filesystem at /mnt/user-data/ that maps to thread-specific directories, ensuring file operations remain scoped and secure regardless of the execution backend.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}