# How to Build a Multi-Provider Fallback Mechanism for Agents in aisuite

> Build a multi-provider fallback for agents with aisuite. Automatically failover between LLM providers to retry requests until one succeeds. Learn how to implement robust agent failover.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: how-to-guide
- Published: 2026-07-30

---

**aisuite enables automatic failover between LLM providers by catching `LLMError` exceptions from provider instances created via `ProviderFactory`, allowing your agents to retry requests with backup providers in sequence until one succeeds.**

aisuite is a Python library that unifies access to multiple large language model providers through a single interface. When building production-grade AI agents, implementing a **multi-provider fallback mechanism** ensures high availability by automatically switching to backup providers when the primary experiences outages, rate limits, or errors. This guide demonstrates how to construct robust fallback logic using aisuite's provider-agnostic architecture as implemented in the `andrewyng/aisuite` repository.

## Understanding the Provider Architecture

aisuite uses a standardized naming convention to route requests. The **model identifier** follows the format `<provider>:<model-name>`, such as `"openai:gpt-4o"` or `"anthropic:claude-3-5-sonnet-20240620"`.

### Dynamic Provider Loading

At runtime, `aisuite.provider.ProviderFactory` dynamically loads the appropriate provider class based on the identifier prefix. According to the source code in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py), the factory locates provider modules following the naming convention `<provider>_provider.py` and instantiates the corresponding `<Provider>Provider` class (e.g., `OpenaiProvider`, `AnthropicProvider`).

Each concrete provider implements the uniform `chat_completions_create` method, ensuring consistent interaction regardless of the underlying SDK. When failures occur, providers raise `aisuite.provider.LLMError`, which serves as the canonical exception for fallback logic to catch.

## Method 1: Simple Fallback Wrapper for Direct Calls

For agents making direct client calls, wrap the provider invocation in a sequencer that iterates through a priority list of models. This approach uses `ProviderFactory.create_provider(provider_key, config)` to instantiate each provider dynamically.

```python
import aisuite as ai
from aisuite.provider import ProviderFactory, LLMError

def chat_with_fallback(model_list, messages, **kwargs):
    """
    Try the models in ``model_list`` in order.
    Returns the first successful response or re-raises the last LLMError.
    """
    last_error = None
    for model in model_list:
        provider_key = model.split(":")[0]                # e.g. "openai"

        provider = ProviderFactory.create_provider(
            provider_key, config={}                       # add any required config here

        )
        try:
            return provider.chat_completions_create(
                model=model, messages=messages, **kwargs
            )
        except LLMError as exc:                           # provider-specific failure

            last_error = exc
    # If we reach here, all providers failed

    raise last_error

```

**Usage example:**

```python
client = ai.Client()

models = [
    "openai:gpt-4o",          # primary

    "anthropic:claude-3-5-sonnet-20240620",  # secondary

    "groq:mixtral-8x7b-32768",               # tertiary

]

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user",   "content": "Summarize the latest news in tech."},
]

response = chat_with_fallback(models, messages, temperature=0.7)
print(response.choices[0].message.content)

```

This helper abstracts provider selection, allowing any part of your agent code to call `chat_with_fallback` instead of the standard `client.chat.completions.create`.

## Method 2: Integrated Fallback via Agent Runner Subclassing

For deeper integration with aisuite's **Agents API**, subclass `aisuite.agents.runner.Runner` to inject fallback logic directly into the execution flow. This method overrides the internal `_call_provider` method used by the `Runner` to make chat requests.

```python
from aisuite.agents.runner import Runner
from aisuite.provider import ProviderFactory, LLMError

class FallbackRunner(Runner):
    def __init__(self, agent, fallback_models, **kwargs):
        super().__init__(agent, **kwargs)
        self.fallback_models = fallback_models   # ordered list of model strings

    def _call_provider(self, model, messages, **kwargs):
        # Override the internal method used by Runner to make a chat request

        for m in self.fallback_models:
            provider_key = m.split(":")[0]
            provider = ProviderFactory.create_provider(provider_key, config={})
            try:
                return provider.chat_completions_create(
                    model=m, messages=messages, **kwargs
                )
            except LLMError:
                continue
        raise RuntimeError("All fallback providers failed.")

```

**Running an agent with fallback:**

```python
import aisuite as ai
from aisuite import Agent

agent = Agent(
    name="news-summarizer",
    model="openai:gpt-4o",        # primary model (fallback logic will ignore this)

    instructions="Summarize tech headlines.",
    tools=[],
)

fallback_models = [
    "openai:gpt-4o",
    "anthropic:claude-3-5-sonnet-20240620",
    "groq:mixtral-8x7b-32768",
]

result = FallbackRunner(agent, fallback_models).run(
    "What are the biggest AI announcements this week?"
)
print(result.final_output)

```

The `FallbackRunner` swaps providers transparently, keeping the agent definition unchanged while ensuring every turn uses the fallback-aware client.

## Method 3: Async Fallback for Concurrent Agents

For asynchronous agent implementations, implement `achat_with_fallback` using the async variant `achat_completions_create`. This follows the same pattern as the synchronous version but awaits coroutines.

```python
import aisuite as ai
from aisuite.provider import ProviderFactory, LLMError

async def achat_with_fallback(models, messages, **kwargs):
    last_error = None
    for model in models:
        provider_key = model.split(":")[0]
        provider = ProviderFactory.create_provider(provider_key, config={})
        try:
            return await provider.achat_completions_create(
                model=model, messages=messages, **kwargs
            )
        except LLMError as exc:
            last_error = exc
    raise last_error

```

Use this function in async agent loops to maintain non-blocking execution while preserving multi-provider resilience.

## Key Source Files Reference

Understanding these core files helps when debugging or extending the fallback mechanism:

- **[`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py)**: Contains the abstract `Provider` class, `ProviderFactory` for dynamic loading, and the `LLMError` exception definition.
- **[`aisuite/providers/openai_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/openai_provider.py)**: Example concrete provider implementing `chat_completions_create` and `achat_completions_create`.
- **[`aisuite/providers/anthropic_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/anthropic_provider.py)**: Demonstrates how multiple providers share the uniform interface.
- **[`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py)**: High-level client that parses the `<provider>:<model>` identifier and forwards calls to the appropriate provider.
- **[`aisuite/agents/runner.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/agents/runner.py)**: Execution engine for agents; designed to be subclassed to inject custom logic like provider fallback.

## Summary

- **aisuite's provider-agnostic design** uses the `<provider>:<model>` identifier format and `ProviderFactory` to enable seamless switching between LLM backends.
- **Catch `LLMError`** from [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) to detect provider-specific failures and trigger fallback logic.
- **Implement fallback** by iterating through a priority list of models, splitting each identifier to extract the provider key, and calling `ProviderFactory.create_provider` to instantiate the correct client.
- **Integrate at the Runner level** by subclassing `aisuite.agents.runner.Runner` and overriding `_call_provider` to make fallback invisible to agent definitions.
- **Support async agents** by using `achat_completions_create` with identical error handling patterns.

## Frequently Asked Questions

### What exception should I catch when implementing a fallback mechanism in aisuite?

Catch `aisuite.provider.LLMError`. This is the unified exception raised by all concrete provider implementations (such as those in [`aisuite/providers/openai_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/openai_provider.py) and [`aisuite/providers/anthropic_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/anthropic_provider.py)) when API calls fail, time out, or return errors. Catching this specific exception allows your fallback logic to distinguish between provider failures and other application errors.

### Can I combine OpenAI and Anthropic models in the same fallback chain?

Yes. Because `ProviderFactory.create_provider` returns instances that share the same interface, you can chain heterogeneous providers like `"openai:gpt-4o"`, `"anthropic:claude-3-5-sonnet-20240620"`, and `"groq:mixtral-8x7b-32768"` in a single fallback list. Each provider handles its own authentication and SDK specifics internally while exposing the uniform `chat_completions_create` method to your agent.

### How do I integrate fallback logic into aisuite's Agent Runner?

Subclass `aisuite.agents.runner.Runner` and override the `_call_provider` method. Inside your override, implement a loop that attempts the request with each provider in your fallback list, catching `LLMError` and continuing to the next provider on failure. This injects fallback behavior at the execution engine level, making it transparent to the agent's definition and instructions.

### Does aisuite support async fallback for non-blocking agent execution?

Yes. aisuite providers implement `achat_completions_create` alongside the synchronous `chat_completions_create`. You can build an async fallback mechanism by awaiting these methods in sequence within an async function, catching `LLMError` exceptions exactly as you would in synchronous code. This allows multi-provider resilience in concurrent or high-throughput async agent architectures.