How to Integrate External LLM Providers with ChatDev: A Step-by-Step Guide

Integrating external LLM providers with ChatDev requires implementing the ModelProvider abstract base class, registering the concrete implementation via ProviderRegistry, and referencing the provider name in your agent YAML configuration.

ChatDev by OpenBMB abstracts LLM interactions behind a clean provider interface, enabling seamless integration of services beyond OpenAI and Gemini. By implementing three core methods and registering your class, you can connect proprietary APIs, open-source models, or specialized cloud services without modifying the framework's core logic. This guide walks through the exact file paths and code patterns used in the ChatDev source to add custom providers like Anthropic Claude, DeepSeek, or Azure OpenAI.

Understanding the Provider Architecture

ChatDev uses a registry-based plugin system located in runtime/node/agent/providers/base.py. The architecture separates concerns into three components:

• ModelProvider (abstract base): Defines the contract each provider must fulfill through create_client(), call_model(), and extract_token_usage() methods.
• ProviderRegistry: A global registry mapping provider names (e.g., openai, gemini) to their concrete classes via ProviderRegistry.register().
• AgentConfig.provider: The configuration enum in entity/configs/node/agent.py that automatically populates valid provider names from the registry snapshot.

When an agent configuration specifies provider: gemini, ChatDev resolves this string to the registered GeminiProvider class and instantiates it with the agent's API credentials.

Implementing a Custom LLM Provider

Step 1 – Create a ModelProvider Subclass

Create a new Python module under runtime/node/agent/providers/. Your class must inherit from ModelProvider and implement three abstract methods:

# runtime/node/agent/providers/custom_provider.py

from typing import Any, List, Optional
from entity.messages import Message
from entity.tool_spec import ToolSpec
from runtime.node.agent import ModelProvider, ModelResponse
from utils.token_tracker import TokenUsage

class CustomProvider(ModelProvider):
    """Adapter for a custom LLM service."""

    def create_client(self):
        """Initialize the vendor SDK with credentials from AgentConfig."""
        import custom_sdk
        client_kwargs = {}
        if self.api_key:
            client_kwargs["api_key"] = self.api_key
        if self.base_url:
            client_kwargs["base_url"] = self.base_url
        return custom_sdk.Client(**client_kwargs)

    def call_model(
        self,
        client,
        conversation: List[Message],
        timeline: List[Any],
        tool_specs: Optional[List[ToolSpec]] = None,
        **kwargs,
    ) -> ModelResponse:
        """Translate ChatDev messages to vendor format and return standardized response."""
        payload = self._build_payload(conversation, tool_specs, kwargs)
        raw_resp = client.chat_completion(**payload)
        self._track_token_usage(raw_resp)
        message = self._deserialize_response(raw_resp)
        return ModelResponse(message=message, raw_response=raw_resp)

    def extract_token_usage(self, response: Any) -> TokenUsage:
        """Extract token counts from vendor response for observability."""
        usage = getattr(response, "usage", {})
        return TokenUsage(
            input_tokens=usage.get("prompt_tokens", 0),
            output_tokens=usage.get("completion_tokens", 0),
            total_tokens=usage.get("total_tokens"),
        )

The create_client() method isolates credential handling, while call_model() handles the request/response translation layer. Implementing extract_token_usage() enables ChatDev's unified token-tracking dashboard.

Step 2 – Register the Provider

Register your class in runtime/node/agent/providers/builtin_providers.py to make it available to the configuration system:

# runtime/node/agent/providers/builtin_providers.py

from runtime.node.agent.providers.base import ProviderRegistry
from runtime.node.agent.providers.custom_provider import CustomProvider

ProviderRegistry.register(
    name="custom",
    provider_class=CustomProvider,
    label="Custom LLM",
    summary="Third-party LLM service supporting ChatDev's tool-calling protocol.",
)

Registration automatically adds the provider name to the AgentConfig.provider enum in entity/configs/node/agent.py, making it a valid option in YAML configurations.

Step 3 – Update Dependencies

Add any required SDK to your environment. Create or modify requirements.txt in your project root:

# requirements.txt

custom-sdk>=1.0.0

Install the dependency with pip install -r requirements.txt before running ChatDev.

Step 4 – Configure the Agent in YAML

Reference the registered provider name in your agent definition:

# yaml_instance/custom_demo.yaml

agents:
  - name: custom_assistant
    provider: custom
    model_name: custom-llm-v1
    api_key: ${CUSTOM_API_KEY}
    base_url: https://api.custom-provider.com/v1
    params:
      temperature: 0.7
      max_output_tokens: 2048
    tooling:
      - name: search_web
        description: "Search the web for information"
        parameters:
          type: object
          properties:
            query:
              type: string

When ChatDev loads this configuration, it instantiates CustomProvider, initializes the client with the provided api_key and base_url, and routes all model calls through your implementation.

Reference Implementation Files

Study these files in the OpenBMB/ChatDev repository to understand production patterns:

File	Purpose
runtime/node/agent/providers/base.py	Abstract ModelProvider class and ProviderRegistry implementation
runtime/node/agent/providers/openai_provider.py	Reference implementation showing chat completions and tool handling
runtime/node/agent/providers/gemini_provider.py	Example of media handling and tool call conversion
runtime/node/agent/providers/builtin_providers.py	Registration hub for built-in providers
entity/configs/node/agent.py	Configuration schema with automatic provider enum generation

Complete Minimal Example

Here is a copy-paste skeleton for rapid prototyping:

# runtime/node/agent/providers/awesome_provider.py

from typing import Any, List, Optional
from entity.messages import Message
from entity.tool_spec import ToolSpec
from runtime.node.agent import ModelProvider, ModelResponse
from utils.token_tracker import TokenUsage

class AwesomeProvider(ModelProvider):
    def create_client(self):
        import awesome_sdk
        return awesome_sdk.Client(api_key=self.api_key)

    def call_model(self, client, conversation, timeline, tool_specs=None, **kwargs):
        messages = [{"role": m.role, "content": m.content} for m in conversation]
        raw = client.chat.completions.create(model=self.model_name, messages=messages)
        return ModelResponse(
            message=Message(role="assistant", content=raw.choices[0].message.content),
            raw_response=raw
        )

    def extract_token_usage(self, response) -> TokenUsage:
        u = getattr(response, "usage", {})
        return TokenUsage(
            input_tokens=u.get("prompt_tokens", 0),
            output_tokens=u.get("completion_tokens", 0)
        )

Register it:

# runtime/node/agent/providers/builtin_providers.py

ProviderRegistry.register(
    name="awesome",
    provider_class=AwesomeProvider,
    label="Awesome AI",
    summary="Example third-party integration"
)

Use it:

agents:
  - name: assistant
    provider: awesome
    model_name: awesome-1.0
    api_key: ${AWESOME_API_KEY}

Summary

• Implement ModelProvider: Create a subclass in runtime/node/agent/providers/ with create_client(), call_model(), and extract_token_usage() methods to handle vendor-specific API logic.
• Register the class: Add a ProviderRegistry.register() call in runtime/node/agent/providers/builtin_providers.py to expose the provider to the configuration system.
• Manage dependencies: Install required SDKs via pip and document them in requirements.txt.
• Configure agents: Reference the provider by its registered name in YAML agent definitions using the provider field.
• Maintain compatibility: Ensure your call_model() implementation handles ChatDev's Message and ToolSpec objects to support the framework's tool-calling and memory features.

Frequently Asked Questions

Can I integrate local LLMs like Ollama or llama.cpp?

Yes. Since the ModelProvider interface only requires a client object returned by create_client(), you can wrap any local inference server. Initialize your client with base_url pointing to localhost and implement call_model() to use the local API schema, following the same pattern as the OpenAI-compatible examples in runtime/node/agent/providers/openai_provider.py.

Do I need to modify ChatDev's core files to add a provider?

No. You only need to create a new file in runtime/node/agent/providers/ and add a registration line to builtin_providers.py. The registry system in runtime/node/agent/providers/base.py ensures that no other part of the codebase requires changes, as agent resolution happens dynamically through ProviderRegistry.get_provider().

How does ChatDev handle tool calling for custom providers?

The call_model() method receives tool_specs: Optional[List[ToolSpec]] containing function schemas. Your implementation must translate these specifications into the vendor's native tool format (e.g., OpenAI's function calling or Gemini's function declarations) and deserialize tool call responses back into ToolCallPayload objects within the returned Message.

Where should I store API keys for external providers?

Store API keys as environment variables and reference them in YAML using the ${ENV_VAR_NAME} syntax, as shown in the configuration examples. The ModelProvider base class automatically exposes self.api_key and self.base_url parsed from the agent configuration, keeping credentials out of version control while making them available to create_client().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}