# How to Implement a Custom LLM Provider in AstrBot: Complete Developer Guide

> Learn how to implement a custom LLM provider in AstrBot by subclassing Provider, implementing abstract methods, and registering your adapter. Follow our developer guide for AstrBot.

- Repository: [AstrBot AI/AstrBot](https://github.com/AstrBotDevs/AstrBot)
- Tags: how-to-guide
- Published: 2026-03-12

---

**To implement a custom LLM provider in AstrBot, subclass the `Provider` base class from [`astrbot/core/provider/provider.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/provider.py), implement the required abstract methods including `text_chat` and `get_models`, register the class using the `@register_provider_adapter` decorator, and add a dynamic import mapping in `ProviderManager.dynamic_import_provider` to make it available via configuration.**

AstrBot loads large language model (LLM) providers through a dynamic plug-in system that supports custom implementations without modifying core framework code. Whether you need to integrate a private API, a local model server, or a niche cloud service, you can implement a custom LLM provider in AstrBot by following the provider abstraction layer defined in the `AstrBotDevs/AstrBot` repository. This guide walks through the architecture, required code structure, and configuration steps using actual source file references.

## Understanding the Provider Architecture

AstrBot's provider system operates through three distinct phases: registration, instantiation, and dynamic mapping. The framework uses abstract base classes to enforce consistent interfaces across all LLM implementations, while the `ProviderManager` handles runtime discovery and lifecycle management.

The core abstractions reside in these key files:

- **[`astrbot/core/provider/provider.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/provider.py)** – Defines the abstract `Provider` class (lines 66-78) that specifies the interface for chat-completion providers, including method signatures for `text_chat`, `get_models`, `get_current_key`, and `set_key`.
- **[`astrbot/core/provider/register.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/register.py)** – Contains the `register_provider_adapter` decorator (lines 14-53) that records provider metadata in `provider_cls_map`, mapping a unique `type` string to your concrete class.
- **[`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py)** – Hosts `ProviderManager` with `dynamic_import_provider` (lines 55-74) and `initialize` (lines 73-82), which dynamically imports modules and instantiates providers based on user configuration.
- **[`astrbot/core/provider/sources/openai_source.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/sources/openai_source.py)** – Provides a reference implementation (`ProviderOpenAIOfficial`) demonstrating the typical workflow for HTTP-based LLM APIs.

When AstrBot starts, the `ProviderManager` reads the provider configuration list, calls `dynamic_import_provider` to map `type` strings to Python modules, instantiates each enabled provider, and stores active instances in `inst_map`.

## Step-by-Step Implementation Guide

### 1. Create the Provider Source File

Create a new Python file under `astrbot/core/provider/sources/`, such as [`my_custom_source.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/my_custom_source.py). This file will contain your concrete implementation and registration decorator. Keep the naming consistent with the module path you'll reference in the dynamic import mapping.

### 2. Inherit from the Provider Base Class

Subclass **`Provider`** (imported from [`astrbot/core/provider/provider.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/provider.py)) to create a chat-completion provider. For other provider types like STT, TTS, or Embeddings, subclass the corresponding abstract class (`STTProvider`, `TTSProvider`, etc.) instead.

Your `__init__` method must accept `provider_config: dict` and `provider_settings: dict` parameters and call `super().__init__()` to ensure proper initialization of the base attributes.

### 3. Implement Required Abstract Methods

You must implement four critical abstract methods defined in the base class (see [`astrbot/core/provider/provider.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/provider.py) lines 66-78):

- **`get_current_key(self) -> str`** – Returns the currently active API key string. The framework calls this to display or rotate keys.
- **`set_key(self, key: str) -> None`** – Accepts a new API key string to support key rotation functionality.
- **`async def get_models(self) -> list[str]`** – Returns a list of model identifiers (e.g., `["gpt-4", "gpt-3.5-turbo"]`) that your provider supports.
- **`async def text_chat(... ) -> LLMResponse`** – The core method that sends the chat request to your LLM backend. The full signature includes parameters for `prompt`, `session_id`, `image_urls`, `contexts`, `system_prompt`, and `tool_calls_result`, and must return an `LLMResponse` object containing the generated content and token usage statistics.

Optionally, implement **`text_chat_stream`** returning `AsyncGenerator[LLMResponse, None]` to support streaming responses.

### 4. Register Your Provider with the Decorator

Apply the **`@register_provider_adapter`** decorator at the module level (imported from [`astrbot/core/provider/register.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/register.py)) to register your class with the framework:

```python
@register_provider_adapter(
    "my_custom_chat_completion",          # Unique type identifier used in config

    "My Custom LLM Provider",            # Human-readable description for WebUI

    provider_type=ProviderType.CHAT_COMPLETION,  # Defaults to chat completion

    default_config_tmpl={                # Optional: default configuration template

        "type": "my_custom_chat_completion",
        "id": "my_custom",
        "enable": False,
        "key": ["${MY_CUSTOM_API_KEY}"],
        "model": "my-model",
        "api_base": "https://api.mycustom.com/v1",
    }
)
class ProviderMyCustom(Provider):
    ...

```

The decorator stores your provider's metadata in `provider_cls_map` as a `ProviderMetaData` object (defined in [`astrbot/core/provider/entities.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/entities.py) lines 49-61), making it discoverable by the manager.

### 5. Wire Up the Dynamic Import

Edit **[`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py)** to add a case statement in `dynamic_import_provider` (around lines 55-74) that maps your `type` string to the module import:

```python
case "my_custom_chat_completion":
    from .sources.my_custom_source import ProviderMyCustom as ProviderMyCustom

```

This mapping allows `ProviderManager` to locate and import your class when the configuration specifies `type: my_custom_chat_completion`.

### 6. Configure and Activate

Add your provider to the AstrBot configuration file (typically [`astrbot.yaml`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot.yaml) or via the WebUI):

```yaml
provider:
  - id: my_custom
    type: my_custom_chat_completion
    enable: true
    key: [ "${MY_CUSTOM_API_KEY}" ]
    model: my-model-v1
    api_base: https://api.mycustom.com/v1/chat
    timeout: 30

provider_settings:
  default_provider_id: my_custom

```

The `ProviderManager.initialize` method (lines 73-82 in [`manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/manager.py)) processes this configuration, instantiates your class with the provided dictionaries, and registers it in `inst_map`. Restart AstrBot or use the `/provider reload` command to activate.

## Complete Working Example

Here is a fully functional skeleton implementing a fictional HTTP-based LLM API:

```python

# File: astrbot/core/provider/sources/my_custom_source.py

from __future__ import annotations

import httpx
from typing import Any

from astrbot.core.provider.provider import Provider
from astrbot.core.provider.entities import LLMResponse, TokenUsage, ToolCallsResult
from astrbot.core.provider.register import register_provider_adapter
from astrbot.core.provider.entities import ProviderType


@register_provider_adapter(
    "my_custom_chat_completion",
    "My Custom LLM Provider",
    provider_type=ProviderType.CHAT_COMPLETION,
)
class ProviderMyCustom(Provider):
    """Minimal implementation of a custom HTTP-based LLM provider."""

    def __init__(self, provider_config: dict, provider_settings: dict) -> None:
        super().__init__(provider_config, provider_settings)
        self._api_key = self.get_keys()[0] or ""
        self._endpoint = provider_config.get("api_base", "https://api.mycustom.com/v1/chat")
        self._timeout = provider_config.get("timeout", 30)

    def get_current_key(self) -> str:
        return self._api_key

    def set_key(self, key: str) -> None:
        self._api_key = key

    async def get_models(self) -> list[str]:
        return ["my-model-v1", "my-model-v2"]

    async def text_chat(
        self,
        prompt: str | None = None,
        session_id: str | None = None,
        image_urls: list[str] | None = None,
        func_tool: Any = None,
        contexts: list[dict] | None = None,
        system_prompt: str | None = None,
        tool_calls_result: ToolCallsResult | None = None,
        model: str | None = None,
        extra_user_content_parts: Any = None,
        **kwargs,
    ) -> LLMResponse:
        payload = {
            "model": model or self.get_model(),
            "messages": contexts or [],
            "max_tokens": kwargs.get("max_tokens", 1024),
        }
        if prompt:
            payload["messages"].append({"role": "user", "content": prompt})
        if system_prompt:
            payload["messages"].insert(0, {"role": "system", "content": system_prompt})

        async with httpx.AsyncClient(timeout=self._timeout) as client:
            resp = await client.post(
                self._endpoint,
                json=payload,
                headers={"Authorization": f"Bearer {self._api_key}"},
            )
            resp.raise_for_status()
            data = resp.json()

        content = data["choices"][0]["message"]["content"]
        usage = TokenUsage(
            input_other=data["usage"]["prompt_tokens"],
            output=data["usage"]["completion_tokens"],
        )
        return LLMResponse(
            role="assistant",
            result_chain=None,
            tools_call_args=[],
            tools_call_name=[],
            tools_call_ids=[],
            reasoning_content="",
            raw_completion=None,
            usage=usage,
        )

```

Add the corresponding import case to [`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py) and the YAML configuration as shown in previous sections to complete the integration.

## Summary

- **Subclass `Provider`** from [`astrbot/core/provider/provider.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/provider.py) and implement `get_current_key`, `set_key`, `get_models`, and `text_chat` to satisfy the abstract interface defined at lines 66-78.
- **Use `@register_provider_adapter`** from [`astrbot/core/provider/register.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/register.py) (lines 14-53) to declare your provider's unique `type` string and metadata.
- **Modify `ProviderManager.dynamic_import_provider`** in [`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py) (lines 55-74) to map the `type` string to your module import path.
- **Return `LLMResponse` objects** from `text_chat` containing content and `TokenUsage` statistics to enable AstrBot's cost tracking and tool-calling features.
- **Configure via YAML** in the `provider` list with your custom `type` identifier; the manager instantiates your class during `initialize` (lines 73-82).

## Frequently Asked Questions

### What abstract methods must I implement for a custom LLM provider in AstrBot?

You must implement four methods defined in [`astrbot/core/provider/provider.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/provider.py) lines 66-78: `get_current_key()` returning the active API key, `set_key(key)` to update the key, `get_models()` returning a list of available model strings, and `text_chat()` returning an `LLMResponse` object. The `text_chat` method must handle the conversation payload, communicate with your LLM backend, and package the response into the required return structure.

### How does AstrBot discover my custom provider at runtime?

AstrBot discovers providers through the `ProviderManager.dynamic_import_provider` method in [`astrbot/core/provider/manager.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/manager.py) (lines 55-74). You must add a `case` statement matching your provider's `type` string that imports your module. The `@register_provider_adapter` decorator registers the class metadata, but the explicit import in the manager is required for Python to load the module containing your class definition.

### Can I implement streaming responses for my custom LLM provider?

Yes, implement the optional **`text_chat_stream`** method returning `AsyncGenerator[LLMResponse, None]`. While the base class requires `text_chat`, streaming support is available by yielding `LLMResponse` chunks as they arrive from your backend API. Check existing implementations in [`astrbot/core/provider/sources/openai_source.py`](https://github.com/AstrBotDevs/AstrBot/blob/main/astrbot/core/provider/sources/openai_source.py) for the streaming pattern using `AsyncGenerator`.

### Where do I configure the API keys and endpoint URL for my custom provider?

Define these in the AstrBot configuration YAML under the `provider` list. Specify the `key` as a list (supporting key rotation), `api_base` for your endpoint URL, and any custom parameters like `timeout` or `model` in your provider's dictionary entry. The `ProviderManager` passes this dictionary as `provider_config` to your class `__init__` method during instantiation (see `ProviderManager.initialize` lines 73-82).