How to Implement a Custom LLM Provider in AstrBot: Complete Developer Guide

To implement a custom LLM provider in AstrBot, subclass the Provider base class from astrbot/core/provider/provider.py, implement the required abstract methods including text_chat and get_models, register the class using the @register_provider_adapter decorator, and add a dynamic import mapping in ProviderManager.dynamic_import_provider to make it available via configuration.

AstrBot loads large language model (LLM) providers through a dynamic plug-in system that supports custom implementations without modifying core framework code. Whether you need to integrate a private API, a local model server, or a niche cloud service, you can implement a custom LLM provider in AstrBot by following the provider abstraction layer defined in the AstrBotDevs/AstrBot repository. This guide walks through the architecture, required code structure, and configuration steps using actual source file references.

Understanding the Provider Architecture

AstrBot's provider system operates through three distinct phases: registration, instantiation, and dynamic mapping. The framework uses abstract base classes to enforce consistent interfaces across all LLM implementations, while the ProviderManager handles runtime discovery and lifecycle management.

The core abstractions reside in these key files:

• astrbot/core/provider/provider.py – Defines the abstract Provider class (lines 66-78) that specifies the interface for chat-completion providers, including method signatures for text_chat, get_models, get_current_key, and set_key.
• astrbot/core/provider/register.py – Contains the register_provider_adapter decorator (lines 14-53) that records provider metadata in provider_cls_map, mapping a unique type string to your concrete class.
• astrbot/core/provider/manager.py – Hosts ProviderManager with dynamic_import_provider (lines 55-74) and initialize (lines 73-82), which dynamically imports modules and instantiates providers based on user configuration.
• astrbot/core/provider/sources/openai_source.py – Provides a reference implementation (ProviderOpenAIOfficial) demonstrating the typical workflow for HTTP-based LLM APIs.

When AstrBot starts, the ProviderManager reads the provider configuration list, calls dynamic_import_provider to map type strings to Python modules, instantiates each enabled provider, and stores active instances in inst_map.

Step-by-Step Implementation Guide

1. Create the Provider Source File

Create a new Python file under astrbot/core/provider/sources/, such as my_custom_source.py. This file will contain your concrete implementation and registration decorator. Keep the naming consistent with the module path you'll reference in the dynamic import mapping.

2. Inherit from the Provider Base Class

Subclass Provider (imported from astrbot/core/provider/provider.py) to create a chat-completion provider. For other provider types like STT, TTS, or Embeddings, subclass the corresponding abstract class (STTProvider, TTSProvider, etc.) instead.

Your __init__ method must accept provider_config: dict and provider_settings: dict parameters and call super().__init__() to ensure proper initialization of the base attributes.

3. Implement Required Abstract Methods

You must implement four critical abstract methods defined in the base class (see astrbot/core/provider/provider.py lines 66-78):

• get_current_key(self) -> str – Returns the currently active API key string. The framework calls this to display or rotate keys.
• set_key(self, key: str) -> None – Accepts a new API key string to support key rotation functionality.
• async def get_models(self) -> list[str] – Returns a list of model identifiers (e.g., ["gpt-4", "gpt-3.5-turbo"]) that your provider supports.
• async def text_chat(... ) -> LLMResponse – The core method that sends the chat request to your LLM backend. The full signature includes parameters for prompt, session_id, image_urls, contexts, system_prompt, and tool_calls_result, and must return an LLMResponse object containing the generated content and token usage statistics.

Optionally, implement text_chat_stream returning AsyncGenerator[LLMResponse, None] to support streaming responses.

4. Register Your Provider with the Decorator

Apply the @register_provider_adapter decorator at the module level (imported from astrbot/core/provider/register.py) to register your class with the framework:

@register_provider_adapter(
    "my_custom_chat_completion",          # Unique type identifier used in config

    "My Custom LLM Provider",            # Human-readable description for WebUI

    provider_type=ProviderType.CHAT_COMPLETION,  # Defaults to chat completion

    default_config_tmpl={                # Optional: default configuration template

        "type": "my_custom_chat_completion",
        "id": "my_custom",
        "enable": False,
        "key": ["${MY_CUSTOM_API_KEY}"],
        "model": "my-model",
        "api_base": "https://api.mycustom.com/v1",
    }
)
class ProviderMyCustom(Provider):
    ...

The decorator stores your provider's metadata in provider_cls_map as a ProviderMetaData object (defined in astrbot/core/provider/entities.py lines 49-61), making it discoverable by the manager.

5. Wire Up the Dynamic Import

Edit astrbot/core/provider/manager.py to add a case statement in dynamic_import_provider (around lines 55-74) that maps your type string to the module import:

case "my_custom_chat_completion":
    from .sources.my_custom_source import ProviderMyCustom as ProviderMyCustom

This mapping allows ProviderManager to locate and import your class when the configuration specifies type: my_custom_chat_completion.

6. Configure and Activate

Add your provider to the AstrBot configuration file (typically astrbot.yaml or via the WebUI):

provider:
  - id: my_custom
    type: my_custom_chat_completion
    enable: true
    key: [ "${MY_CUSTOM_API_KEY}" ]
    model: my-model-v1
    api_base: https://api.mycustom.com/v1/chat
    timeout: 30

provider_settings:
  default_provider_id: my_custom

The ProviderManager.initialize method (lines 73-82 in manager.py) processes this configuration, instantiates your class with the provided dictionaries, and registers it in inst_map. Restart AstrBot or use the /provider reload command to activate.

Complete Working Example

Here is a fully functional skeleton implementing a fictional HTTP-based LLM API:

# File: astrbot/core/provider/sources/my_custom_source.py

from __future__ import annotations

import httpx
from typing import Any

from astrbot.core.provider.provider import Provider
from astrbot.core.provider.entities import LLMResponse, TokenUsage, ToolCallsResult
from astrbot.core.provider.register import register_provider_adapter
from astrbot.core.provider.entities import ProviderType


@register_provider_adapter(
    "my_custom_chat_completion",
    "My Custom LLM Provider",
    provider_type=ProviderType.CHAT_COMPLETION,
)
class ProviderMyCustom(Provider):
    """Minimal implementation of a custom HTTP-based LLM provider."""

    def __init__(self, provider_config: dict, provider_settings: dict) -> None:
        super().__init__(provider_config, provider_settings)
        self._api_key = self.get_keys()[0] or ""
        self._endpoint = provider_config.get("api_base", "https://api.mycustom.com/v1/chat")
        self._timeout = provider_config.get("timeout", 30)

    def get_current_key(self) -> str:
        return self._api_key

    def set_key(self, key: str) -> None:
        self._api_key = key

    async def get_models(self) -> list[str]:
        return ["my-model-v1", "my-model-v2"]

    async def text_chat(
        self,
        prompt: str | None = None,
        session_id: str | None = None,
        image_urls: list[str] | None = None,
        func_tool: Any = None,
        contexts: list[dict] | None = None,
        system_prompt: str | None = None,
        tool_calls_result: ToolCallsResult | None = None,
        model: str | None = None,
        extra_user_content_parts: Any = None,
        **kwargs,
    ) -> LLMResponse:
        payload = {
            "model": model or self.get_model(),
            "messages": contexts or [],
            "max_tokens": kwargs.get("max_tokens", 1024),
        }
        if prompt:
            payload["messages"].append({"role": "user", "content": prompt})
        if system_prompt:
            payload["messages"].insert(0, {"role": "system", "content": system_prompt})

        async with httpx.AsyncClient(timeout=self._timeout) as client:
            resp = await client.post(
                self._endpoint,
                json=payload,
                headers={"Authorization": f"Bearer {self._api_key}"},
            )
            resp.raise_for_status()
            data = resp.json()

        content = data["choices"][0]["message"]["content"]
        usage = TokenUsage(
            input_other=data["usage"]["prompt_tokens"],
            output=data["usage"]["completion_tokens"],
        )
        return LLMResponse(
            role="assistant",
            result_chain=None,
            tools_call_args=[],
            tools_call_name=[],
            tools_call_ids=[],
            reasoning_content="",
            raw_completion=None,
            usage=usage,
        )

Add the corresponding import case to astrbot/core/provider/manager.py and the YAML configuration as shown in previous sections to complete the integration.

Summary

• Subclass Provider from astrbot/core/provider/provider.py and implement get_current_key, set_key, get_models, and text_chat to satisfy the abstract interface defined at lines 66-78.
• Use @register_provider_adapter from astrbot/core/provider/register.py (lines 14-53) to declare your provider's unique type string and metadata.
• Modify ProviderManager.dynamic_import_provider in astrbot/core/provider/manager.py (lines 55-74) to map the type string to your module import path.
• Return LLMResponse objects from text_chat containing content and TokenUsage statistics to enable AstrBot's cost tracking and tool-calling features.
• Configure via YAML in the provider list with your custom type identifier; the manager instantiates your class during initialize (lines 73-82).

Frequently Asked Questions

What abstract methods must I implement for a custom LLM provider in AstrBot?

You must implement four methods defined in astrbot/core/provider/provider.py lines 66-78: get_current_key() returning the active API key, set_key(key) to update the key, get_models() returning a list of available model strings, and text_chat() returning an LLMResponse object. The text_chat method must handle the conversation payload, communicate with your LLM backend, and package the response into the required return structure.

How does AstrBot discover my custom provider at runtime?

AstrBot discovers providers through the ProviderManager.dynamic_import_provider method in astrbot/core/provider/manager.py (lines 55-74). You must add a case statement matching your provider's type string that imports your module. The @register_provider_adapter decorator registers the class metadata, but the explicit import in the manager is required for Python to load the module containing your class definition.

Can I implement streaming responses for my custom LLM provider?

Yes, implement the optional text_chat_stream method returning AsyncGenerator[LLMResponse, None]. While the base class requires text_chat, streaming support is available by yielding LLMResponse chunks as they arrive from your backend API. Check existing implementations in astrbot/core/provider/sources/openai_source.py for the streaming pattern using AsyncGenerator.

Where do I configure the API keys and endpoint URL for my custom provider?

Define these in the AstrBot configuration YAML under the provider list. Specify the key as a list (supporting key rotation), api_base for your endpoint URL, and any custom parameters like timeout or model in your provider's dictionary entry. The ProviderManager passes this dictionary as provider_config to your class __init__ method during instantiation (see ProviderManager.initialize lines 73-82).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}