How to Integrate Multiple LLM Providers (Azure, Ollama, Anthropic) with MetaGPT

MetaGPT abstracts LLM interactions through a provider registry that lets you switch between Azure, Ollama, Anthropic, and other services by changing a YAML configuration file, without modifying agent code.

MetaGPT is a multi-agent framework that requires flexible LLM backend support for different deployment scenarios. Whether you need Azure's enterprise compliance, Ollama's local privacy, or Anthropic's Claude models, integrating multiple LLM providers with MetaGPT follows a consistent registry-based pattern defined in the metagpt/provider directory.

Understanding MetaGPT's LLM Provider Architecture

MetaGPT's LLM integration relies on three core abstractions: configuration objects, a provider registry, and a uniform async interface. The system is designed so that agents call generic methods like aask() while the underlying provider handles service-specific HTTP implementations.

The Provider Registry Pattern

The LLMProviderRegistry in metagpt/provider/llm_provider_registry.py maintains a global mapping between LLMType enum values and concrete provider classes. Each provider implementation registers itself using the @register_provider decorator at import time.

For example, the Azure provider in metagpt/provider/azure_openai_api.py registers itself with:

@register_provider(LLMType.AZURE)
class AzureOpenAILLM(BaseLLM):
    ...

Core Components

Component	File Path	Purpose
LLMConfig	metagpt/configs/llm_config.py	Validates and stores API keys, endpoints, models, and provider types
LLMType	metagpt/configs/llm_config.py	Enum defining supported providers (OPENAI, AZURE, OLLAMA, ANTHROPIC, etc.)
BaseLLM	metagpt/provider/base_llm.py	Abstract class defining aask(), acompletion(), and shared utilities
LLM() factory	metagpt/llm.py	Public entry point that instantiates the correct provider based on config

Configuring Azure, Ollama, and Anthropic Providers

MetaGPT uses YAML configuration files to define provider-specific settings. The api_type field determines which provider class gets instantiated by the factory.

Azure OpenAI Configuration

Create a YAML file for Azure deployments:

# config_azure.yaml

api_type: azure
api_key: YOUR_AZURE_KEY
base_url: https://YOUR_RESOURCE_NAME.openai.azure.com/
api_version: 2023-05-15
model: gpt-4o
temperature: 0.2
stream: true

The AzureOpenAILLM class in metagpt/provider/azure_openai_api.py handles Azure-specific authentication and endpoint construction.

Ollama Local Deployment

For local LLM hosting with Ollama:

# config_ollama.yaml

api_type: ollama
base_url: http://localhost:11434
model: llama3:8b
temperature: 0.7
stream: true

The OllamaLLM implementation in metagpt/provider/ollama_api.py supports chat, generate, and embedding endpoints for self-hosted models.

Anthropic Claude Setup

To use Anthropic's Claude models:

# config_anthropic.yaml

api_type: anthropic
api_key: sk-ant-...
base_url: https://api.anthropic.com
model: claude-3-5-sonnet-20240620
temperature: 0.0
stream: true

The AnthropicLLM class in metagpt/provider/anthropic_api.py implements the Anthropic Messages API with proper message formatting and token counting.

Switching Between LLM Providers in Code

The LLM() factory in metagpt/llm.py provides a unified entry point for instantiating any registered provider. Import the factory and configuration classes:

from metagpt.llm import LLM
from metagpt.configs.llm_config import LLMConfig

# Load Azure configuration

config = LLMConfig.parse_file("config_azure.yaml")
azure_llm = LLM(config)  # Returns AzureOpenAILLM instance

# Use the uniform async interface

response = await azure_llm.aask("Explain quantum computing in simple terms.")
print(response)

Switching to Ollama or Anthropic requires only changing the configuration file:

# Switch to local Ollama instance

ollama_cfg = LLMConfig.parse_file("config_ollama.yaml")
ollama_llm = LLM(ollama_cfg)

# Switch to Anthropic Claude

anthropic_cfg = LLMConfig.parse_file("config_anthropic.yaml")
anthropic_llm = LLM(anthropic_cfg)

All providers expose the same methods: aask(), acompletion(), aask_code(), and get_choice_text(), ensuring agents remain agnostic to the underlying service.

Extending MetaGPT with Custom LLM Providers

You can add support for new LLM services by implementing the BaseLLM interface and registering the class with the provider registry.

Create a new file in metagpt/provider/my_custom_api.py:

from metagpt.provider.base_llm import BaseLLM
from metagpt.provider.llm_provider_registry import register_provider
from metagpt.configs.llm_config import LLMType

@register_provider(LLMType.OPEN_LLM)  # Use existing enum or add a new one

class MyCustomLLM(BaseLLM):
    def __init__(self, config):
        super().__init__(config)
        self.model = config.model
        # Initialize your HTTP client here

        
    async def _achat_completion(self, messages, timeout=60):
        # Implement the actual API call to your LLM service

        # Must return a response object compatible with get_choice_text()

        pass
        
    def get_choice_text(self, response) -> str:
        # Extract the text content from your API response

        return response["choices"][0]["message"]["content"]

After placing myprovider.py in metagpt/provider/, the registry automatically knows about MyCustomLLM. Use it by setting api_type: open_llm (or a new enum entry) in the config.

Summary

• MetaGPT uses a provider registry pattern to abstract LLM interactions, defined in metagpt/provider/llm_provider_registry.py and metagpt/llm.py.
• Configuration-driven switching allows you to change between Azure, Ollama, Anthropic, and other providers by modifying YAML files without touching agent code.
• Uniform async API across all providers (aask, acompletion, aask_code) ensures agents remain agnostic to the underlying service implementation in files like metagpt/provider/base_llm.py.
• Extensible architecture lets you add custom providers by subclassing BaseLLM and using the @register_provider decorator, enabling integration with self-hosted or specialized LLM services.

Frequently Asked Questions

How do I switch between LLM providers without restarting my MetaGPT application?

MetaGPT instantiates LLM providers through the LLM() factory in metagpt/llm.py based on the LLMConfig object passed at initialization. To switch providers dynamically, create a new configuration object pointing to a different provider and instantiate a fresh LLM instance. Agents can accept this new instance via their constructor, or you can implement a factory pattern in your application code to manage provider switching at runtime without restarting the entire MetaGPT process.

Does MetaGPT support streaming responses from providers like Ollama and Anthropic?

Yes, the BaseLLM class in metagpt/provider/base_llm.py defines async methods that support streaming. When you set stream: true in your YAML configuration, providers like OllamaLLM in metagpt/provider/ollama_api.py and AnthropicLLM in metagpt/provider/anthropic_api.py handle streaming responses appropriately. The uniform interface ensures that agent code consuming aask() or acompletion() works identically whether streaming is enabled or not.

Can I use different LLM providers for different agents in the same MetaGPT project?

Absolutely. Since the LLM instance is typically passed to agent constructors or roles, you can instantiate multiple LLM objects with different LLMConfig configurations and assign them to different agents. For example, you might use Azure OpenAI for the ProductManager role requiring high reliability, while using a local Ollama instance for the Engineer role to keep code generation costs low. Each agent operates independently with its configured provider via the common interface defined in metagpt/provider/base_llm.py.

What is the minimum implementation required to add a new custom LLM provider to MetaGPT?

To add a new provider, create a class that inherits from BaseLLM in metagpt/provider/base_llm.py, implement the _achat_completion() method to handle the actual API call, and decorate the class with @register_provider(LLMType.YOUR_TYPE) from metagpt/provider/llm_provider_registry.py. You must also implement get_choice_text() to extract response content. Optionally, override other methods like acompletion() if your API requires special handling for non-chat completions. After implementation, set the corresponding api_type in your YAML configuration to activate the new provider.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}