How to Configure Model Provider Settings and Switch Between OpenAI, LiteLLM, and Custom Providers in openai-agents-python

Use the model_provider field in RunConfig to specify a provider instance, or rely on the default MultiProvider which routes requests based on prefixes like litellm/ or any-llm/ to switch between OpenAI, LiteLLM, and custom backends without changing code structure.

The openai-agents-python framework decouples agent logic from LLM backends through a provider abstraction. To configure model provider settings, you modify the RunConfig object passed to Runner.run(), setting either a specific provider instance or utilizing the prefix-based routing system defined in src/agents/run_config.py.

Understanding the ModelProvider Architecture

The framework centers on the ModelProvider interface, which acts as a factory for creating Model instances capable of executing completions.

The ModelProvider Interface

In src/agents/models/interface.py, the abstract base class defines the contract:

class ModelProvider(ABC):
    @abstractmethod
    def get_model(self, model_name: str | None) -> Model:
        ...

Any custom provider must implement get_model() to return a concrete Model implementation. The framework uses this method to resolve model names into executable backends during agent runs.

MultiProvider Routing Logic

The default RunConfig.model_provider is an instance of MultiProvider defined in src/agents/models/multi_provider.py. This class maintains a prefix-to-provider mapping that enables automatic routing based on model name conventions.

The constructor accepts a provider_map argument (lines 73-78) that defines which prefixes route to which provider implementations. By default, the framework includes built-in mappings for OpenAI, LiteLLM, and Any-LLM.

Switching Between Built-in Providers

You can switch between supported backends without modifying provider code by using standardized prefixes in your model names.

Using OpenAI (Default)

When no prefix is provided, or when using the openai/ prefix, the framework routes to OpenAIProvider in src/agents/models/openai_provider.py. This provider creates either OpenAIChatCompletionsModel or OpenAIResponsesModel instances depending on your configuration.

from agents import Runner, Agent, RunConfig

agent = Agent(name="Assistant", instructions="You are helpful.")

# Both calls route to OpenAIProvider

run_cfg_default = RunConfig(model="gpt-4o")
run_cfg_explicit = RunConfig(model="openai/gpt-4o")

Using LiteLLM with Prefix Routing

To route requests through LiteLLM, prepend litellm/ to your model name. The MultiProvider detects this prefix and delegates to LitellmProvider in src/agents/extensions/models/litellm_provider.py.

from agents import Runner, Agent, RunConfig

agent = Agent(name="Assistant", instructions="You are helpful.")

# Routes to LitellmProvider via prefix detection

run_cfg = RunConfig(model="litellm/openai/gpt-4o")

result = await Runner.run(agent, "Tell me a joke.", run_config=run_cfg)

The LitellmProvider.get_model() method (lines 22-24) returns a LitellmModel instance that wraps the LiteLLM completion interface.

Using Any-LLM Providers

For the any-llm ecosystem, use the any-llm/ prefix. This routes to AnyLLMProvider in src/agents/extensions/models/any_llm_provider.py, which supports providers like OpenRouter.

run_cfg = RunConfig(model="any-llm/openrouter/openai/gpt-4o")

Implementing Custom Model Providers

When built-in providers don't meet your requirements, implement a custom ModelProvider to integrate proprietary APIs or self-hosted endpoints.

Creating a Custom Provider Class

Create a class inheriting from ModelProvider and implement get_model(). The example in examples/model_providers/custom_example_provider.py demonstrates this pattern:

import os
from openai import AsyncOpenAI
from agents import ModelProvider, Model, OpenAIChatCompletionsModel

class CustomModelProvider(ModelProvider):
    def __init__(self, base_url: str, api_key: str):
        self.client = AsyncOpenAI(base_url=base_url, api_key=api_key)
    
    def get_model(self, model_name: str | None) -> Model:
        return OpenAIChatCompletionsModel(
            model=model_name or "gpt-4o",
            openai_client=self.client,
        )

Integrating Custom Providers with RunConfig

Pass your provider instance directly to RunConfig.model_provider to bypass the default MultiProvider routing:

from agents import Runner, Agent, RunConfig, set_tracing_disabled

# Initialize your custom provider

custom_provider = CustomModelProvider(
    base_url=os.getenv("EXAMPLE_BASE_URL"),
    api_key=os.getenv("EXAMPLE_API_KEY"),
)

# Configure the run to use your provider

run_cfg = RunConfig(
    model_provider=custom_provider,
    model="gpt-4o"
)

agent = Agent(name="Assistant", instructions="Speak in haiku.")
set_tracing_disabled(disabled=True)  # Optional: disable tracing if needed

result = await Runner.run(agent, "Describe sunrise.", run_config=run_cfg)

This approach gives you full control over the HTTP client, authentication, and endpoint configuration while maintaining compatibility with the agent framework.

Advanced MultiProvider Configuration

For scenarios requiring multiple custom providers alongside built-in ones, extend MultiProvider rather than replacing it.

from agents.models.multi_provider import MultiProvider, MultiProviderMap
from agents.extensions.models.litellm_provider import LitellmProvider

# Suppose you have a vendor that expects the prefix "myvendor/"

class MyVendorProvider(ModelProvider):
    def get_model(self, model_name: str | None) -> Model:
        # Build and return a Model that knows how to call your vendor's API

        ...

# Build a map that adds the new prefix

provider_map = MultiProviderMap()
provider_map.add_provider("myvendor", MyVendorProvider())
provider_map.add_provider("litellm", LitellmProvider())  # Reuse built-in

# Initialize MultiProvider with custom map

custom_multi = MultiProvider(provider_map=provider_map)

# Use in RunConfig

run_cfg = RunConfig(model_provider=custom_multi, model="myvendor/awesome-llm")

The MultiProvider.__init__ method (lines 73-78 in src/agents/models/multi_provider.py) accepts this provider_map to build the routing table. Now any model name starting with myvendor/ routes to your custom provider while preserving access to litellm/ and default OpenAI models.

Summary

• RunConfig in src/agents/run_config.py controls model provider selection through the model_provider field.

• Default routing uses MultiProvider, which automatically routes model names based on prefixes:

  • No prefix or openai/ → OpenAIProvider
  • litellm/ → LitellmProvider
  • any-llm/ → AnyLLMProvider

• Custom providers implement the ModelProvider interface from src/agents/models/interface.py and can be passed directly to RunConfig to bypass prefix routing.

• Advanced configurations can extend MultiProvider with custom MultiProviderMap instances to add new prefixes while retaining built-in provider support.

Frequently Asked Questions

How do I switch from OpenAI to LiteLLM without changing my agent code?

Add the litellm/ prefix to your model name in RunConfig. Pass RunConfig(model="litellm/...") to Runner.run(). The MultiProvider detects the prefix and automatically routes to LitellmProvider in src/agents/extensions/models/litellm_provider.py without requiring any changes to your agent definition or instructions.

Can I use a self-hosted OpenAI-compatible API with openai-agents-python?

Yes. Create a custom ModelProvider that returns an OpenAIChatCompletionsModel initialized with a custom AsyncOpenAI client pointing to your base URL. Pass this provider to RunConfig.model_provider. This pattern is demonstrated in examples/model_providers/custom_example_provider.py and allows you to route all model calls to your private endpoint while maintaining the OpenAI-compatible chat completions interface.

What is the difference between ModelProvider and Model in the framework?

ModelProvider is a factory interface defined in src/agents/models/interface.py that creates Model instances. Model is the concrete class that implements the actual inference logic, such as OpenAIChatCompletionsModel or LitellmModel. The provider abstraction allows the framework to switch between different LLM backends during Runner.run() execution without changing the agent's core logic or conversation management.

How do I add routing for a new custom provider without replacing the default MultiProvider?

Instantiate a MultiProviderMap, add your custom provider with a unique prefix using add_provider(), and pass this map to the MultiProvider constructor via the provider_map argument. Then use this customized MultiProvider instance in your RunConfig. This preserves built-in prefixes like openai/ and litellm/ while adding your custom routing rules, as implemented in src/agents/models/multi_provider.py lines 73-78.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}