# How to Integrate External LLM Providers with ChatDev: A Step-by-Step Guide

> Integrate external LLM providers with ChatDev using ModelProvider and ProviderRegistry. Customize your AI assistant beyond OpenAI with this step by step guide.

- Repository: [OpenBMB/ChatDev](https://github.com/OpenBMB/ChatDev)
- Tags: how-to-guide
- Published: 2026-04-01

---

**Integrating external LLM providers with ChatDev requires implementing the `ModelProvider` abstract base class, registering the concrete implementation via `ProviderRegistry`, and referencing the provider name in your agent YAML configuration.**

ChatDev by OpenBMB abstracts LLM interactions behind a clean provider interface, enabling seamless integration of services beyond OpenAI and Gemini. By implementing three core methods and registering your class, you can connect proprietary APIs, open-source models, or specialized cloud services without modifying the framework's core logic. This guide walks through the exact file paths and code patterns used in the ChatDev source to add custom providers like Anthropic Claude, DeepSeek, or Azure OpenAI.

## Understanding the Provider Architecture

ChatDev uses a registry-based plugin system located in [`runtime/node/agent/providers/base.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/base.py). The architecture separates concerns into three components:

- **`ModelProvider`** (abstract base): Defines the contract each provider must fulfill through `create_client()`, `call_model()`, and `extract_token_usage()` methods.
- **`ProviderRegistry`**: A global registry mapping provider names (e.g., `openai`, `gemini`) to their concrete classes via `ProviderRegistry.register()`.
- **`AgentConfig.provider`**: The configuration enum in [`entity/configs/node/agent.py`](https://github.com/OpenBMB/ChatDev/blob/main/entity/configs/node/agent.py) that automatically populates valid provider names from the registry snapshot.

When an agent configuration specifies `provider: gemini`, ChatDev resolves this string to the registered `GeminiProvider` class and instantiates it with the agent's API credentials.

## Implementing a Custom LLM Provider

### Step 1 – Create a ModelProvider Subclass

Create a new Python module under `runtime/node/agent/providers/`. Your class must inherit from `ModelProvider` and implement three abstract methods:

```python

# runtime/node/agent/providers/custom_provider.py

from typing import Any, List, Optional
from entity.messages import Message
from entity.tool_spec import ToolSpec
from runtime.node.agent import ModelProvider, ModelResponse
from utils.token_tracker import TokenUsage

class CustomProvider(ModelProvider):
    """Adapter for a custom LLM service."""

    def create_client(self):
        """Initialize the vendor SDK with credentials from AgentConfig."""
        import custom_sdk
        client_kwargs = {}
        if self.api_key:
            client_kwargs["api_key"] = self.api_key
        if self.base_url:
            client_kwargs["base_url"] = self.base_url
        return custom_sdk.Client(**client_kwargs)

    def call_model(
        self,
        client,
        conversation: List[Message],
        timeline: List[Any],
        tool_specs: Optional[List[ToolSpec]] = None,
        **kwargs,
    ) -> ModelResponse:
        """Translate ChatDev messages to vendor format and return standardized response."""
        payload = self._build_payload(conversation, tool_specs, kwargs)
        raw_resp = client.chat_completion(**payload)
        self._track_token_usage(raw_resp)
        message = self._deserialize_response(raw_resp)
        return ModelResponse(message=message, raw_response=raw_resp)

    def extract_token_usage(self, response: Any) -> TokenUsage:
        """Extract token counts from vendor response for observability."""
        usage = getattr(response, "usage", {})
        return TokenUsage(
            input_tokens=usage.get("prompt_tokens", 0),
            output_tokens=usage.get("completion_tokens", 0),
            total_tokens=usage.get("total_tokens"),
        )

```

The `create_client()` method isolates credential handling, while `call_model()` handles the request/response translation layer. Implementing `extract_token_usage()` enables ChatDev's unified token-tracking dashboard.

### Step 2 – Register the Provider

Register your class in [`runtime/node/agent/providers/builtin_providers.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/builtin_providers.py) to make it available to the configuration system:

```python

# runtime/node/agent/providers/builtin_providers.py

from runtime.node.agent.providers.base import ProviderRegistry
from runtime.node.agent.providers.custom_provider import CustomProvider

ProviderRegistry.register(
    name="custom",
    provider_class=CustomProvider,
    label="Custom LLM",
    summary="Third-party LLM service supporting ChatDev's tool-calling protocol.",
)

```

Registration automatically adds the provider name to the `AgentConfig.provider` enum in [`entity/configs/node/agent.py`](https://github.com/OpenBMB/ChatDev/blob/main/entity/configs/node/agent.py), making it a valid option in YAML configurations.

### Step 3 – Update Dependencies

Add any required SDK to your environment. Create or modify [`requirements.txt`](https://github.com/OpenBMB/ChatDev/blob/main/requirements.txt) in your project root:

```bash

# requirements.txt

custom-sdk>=1.0.0

```

Install the dependency with `pip install -r requirements.txt` before running ChatDev.

### Step 4 – Configure the Agent in YAML

Reference the registered provider name in your agent definition:

```yaml

# yaml_instance/custom_demo.yaml

agents:
  - name: custom_assistant
    provider: custom
    model_name: custom-llm-v1
    api_key: ${CUSTOM_API_KEY}
    base_url: https://api.custom-provider.com/v1
    params:
      temperature: 0.7
      max_output_tokens: 2048
    tooling:
      - name: search_web
        description: "Search the web for information"
        parameters:
          type: object
          properties:
            query:
              type: string

```

When ChatDev loads this configuration, it instantiates `CustomProvider`, initializes the client with the provided `api_key` and `base_url`, and routes all model calls through your implementation.

## Reference Implementation Files

Study these files in the OpenBMB/ChatDev repository to understand production patterns:

| File | Purpose |
|------|---------|
| [`runtime/node/agent/providers/base.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/base.py) | Abstract `ModelProvider` class and `ProviderRegistry` implementation |
| [`runtime/node/agent/providers/openai_provider.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/openai_provider.py) | Reference implementation showing chat completions and tool handling |
| [`runtime/node/agent/providers/gemini_provider.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/gemini_provider.py) | Example of media handling and tool call conversion |
| [`runtime/node/agent/providers/builtin_providers.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/builtin_providers.py) | Registration hub for built-in providers |
| [`entity/configs/node/agent.py`](https://github.com/OpenBMB/ChatDev/blob/main/entity/configs/node/agent.py) | Configuration schema with automatic provider enum generation |

## Complete Minimal Example

Here is a copy-paste skeleton for rapid prototyping:

```python

# runtime/node/agent/providers/awesome_provider.py

from typing import Any, List, Optional
from entity.messages import Message
from entity.tool_spec import ToolSpec
from runtime.node.agent import ModelProvider, ModelResponse
from utils.token_tracker import TokenUsage

class AwesomeProvider(ModelProvider):
    def create_client(self):
        import awesome_sdk
        return awesome_sdk.Client(api_key=self.api_key)

    def call_model(self, client, conversation, timeline, tool_specs=None, **kwargs):
        messages = [{"role": m.role, "content": m.content} for m in conversation]
        raw = client.chat.completions.create(model=self.model_name, messages=messages)
        return ModelResponse(
            message=Message(role="assistant", content=raw.choices[0].message.content),
            raw_response=raw
        )

    def extract_token_usage(self, response) -> TokenUsage:
        u = getattr(response, "usage", {})
        return TokenUsage(
            input_tokens=u.get("prompt_tokens", 0),
            output_tokens=u.get("completion_tokens", 0)
        )

```

Register it:

```python

# runtime/node/agent/providers/builtin_providers.py

ProviderRegistry.register(
    name="awesome",
    provider_class=AwesomeProvider,
    label="Awesome AI",
    summary="Example third-party integration"
)

```

Use it:

```yaml
agents:
  - name: assistant
    provider: awesome
    model_name: awesome-1.0
    api_key: ${AWESOME_API_KEY}

```

## Summary

- **Implement `ModelProvider`**: Create a subclass in `runtime/node/agent/providers/` with `create_client()`, `call_model()`, and `extract_token_usage()` methods to handle vendor-specific API logic.
- **Register the class**: Add a `ProviderRegistry.register()` call in [`runtime/node/agent/providers/builtin_providers.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/builtin_providers.py) to expose the provider to the configuration system.
- **Manage dependencies**: Install required SDKs via `pip` and document them in [`requirements.txt`](https://github.com/OpenBMB/ChatDev/blob/main/requirements.txt).
- **Configure agents**: Reference the provider by its registered name in YAML agent definitions using the `provider` field.
- **Maintain compatibility**: Ensure your `call_model()` implementation handles ChatDev's `Message` and `ToolSpec` objects to support the framework's tool-calling and memory features.

## Frequently Asked Questions

### Can I integrate local LLMs like Ollama or llama.cpp?

Yes. Since the `ModelProvider` interface only requires a client object returned by `create_client()`, you can wrap any local inference server. Initialize your client with `base_url` pointing to `localhost` and implement `call_model()` to use the local API schema, following the same pattern as the OpenAI-compatible examples in [`runtime/node/agent/providers/openai_provider.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/openai_provider.py).

### Do I need to modify ChatDev's core files to add a provider?

No. You only need to create a new file in `runtime/node/agent/providers/` and add a registration line to [`builtin_providers.py`](https://github.com/OpenBMB/ChatDev/blob/main/builtin_providers.py). The registry system in [`runtime/node/agent/providers/base.py`](https://github.com/OpenBMB/ChatDev/blob/main/runtime/node/agent/providers/base.py) ensures that no other part of the codebase requires changes, as agent resolution happens dynamically through `ProviderRegistry.get_provider()`.

### How does ChatDev handle tool calling for custom providers?

The `call_model()` method receives `tool_specs: Optional[List[ToolSpec]]` containing function schemas. Your implementation must translate these specifications into the vendor's native tool format (e.g., OpenAI's function calling or Gemini's function declarations) and deserialize tool call responses back into `ToolCallPayload` objects within the returned `Message`.

### Where should I store API keys for external providers?

Store API keys as environment variables and reference them in YAML using the `${ENV_VAR_NAME}` syntax, as shown in the configuration examples. The `ModelProvider` base class automatically exposes `self.api_key` and `self.base_url` parsed from the agent configuration, keeping credentials out of version control while making them available to `create_client()`.