# How to Build a Multi-Provider Fallback Mechanism Using aisuite

> Learn to build a multi-provider fallback mechanism with aisuite. Implement a custom FallbackProvider to automatically switch to the next available LLM backend on error.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: tutorial
- Published: 2026-07-27

---

**You can build a multi-provider fallback mechanism using aisuite by implementing a custom `FallbackProvider` class that wraps the `ProviderFactory`, iterates over an ordered list of backends, and catches `LLMError` to automatically delegate to the next available provider.**

The `andrewyng/aisuite` library abstracts LLM and audio backends behind a single `Provider` interface, which makes it easy to construct a multi-provider fallback mechanism using aisuite without touching provider-specific internals. Every concrete implementation—from `OpenAIProvider` to `AnthropicProvider` and `GeminiProvider`—lives in `aisuite/providers/*_provider.py` and inherits the same methods defined in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py).

## aisuite Provider Architecture

### The `Provider` Abstract Base Class

In [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py), the `Provider` class defines the core contract that every backend must implement. Concrete providers override methods such as `chat_completions_create`, `achat_completions_create`, and `chat_completions_create_stream`. Because all providers share this uniform interface, a wrapper can delegate any call without knowing which backend is running underneath.

### `ProviderFactory` Dynamic Loading

The `ProviderFactory` class in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) handles dynamic imports using a naming convention. Calling `ProviderFactory.create_provider("openai", config)` lazily loads the corresponding module and instantiates the class. This factory accepts short string keys—such as `"openai"`, `"anthropic"`, or `"gemini"`—and a configuration dictionary, then returns a fully initialized provider object.

### The Client Provider Registry

The central client in [`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py) maintains a mapping called `self.providers` that holds active provider instances. When you register a new provider instance under a custom key, the rest of aisuite treats it like any native backend. This registry is what lets you drop a fallback wrapper into the stack transparently.

## Implementing the `FallbackProvider` Wrapper

To create a **multi-provider fallback mechanism using aisuite**, follow three steps:

1. **Define an ordered list of provider keys** from primary to secondary.
2. **Create a thin wrapper** that instantiates each provider via `ProviderFactory`.
3. **Delegate every call** through a helper that catches `LLMError` and retries the next provider.

### Wrapper Class Implementation

```python
from aisuite.provider import Provider, ProviderFactory, LLMError
from typing import List, Dict, Any


class FallbackProvider(Provider):
    """Wraps several providers and falls back on error."""
    def __init__(self, provider_keys: List[str], configs: List[Dict[str, Any]]):
        super().__init__()
        if len(provider_keys) != len(configs):
            raise ValueError("Keys and configs must have the same length")
        self._providers: List[Provider] = [
            ProviderFactory.create_provider(k, cfg)
            for k, cfg in zip(provider_keys, configs)
        ]

    def _run_with_fallback(self, method_name: str, *args, **kwargs):
        last_exc = None
        for prov in self._providers:
            try:
                method = getattr(prov, method_name)
                return method(*args, **kwargs)
            except LLMError as exc:
                last_exc = exc
        raise last_exc or LLMError("All providers failed")

    def chat_completions_create(self, model, messages, **kwargs):
        return self._run_with_fallback(
            "chat_completions_create", model, messages, **kwargs
        )

    async def achat_completions_create(self, model, messages, **kwargs):
        return await self._run_with_fallback(
            "achat_completions_create", model, messages, **kwargs
        )

    def chat_completions_create_stream(self, model, messages, **kwargs):
        return self._run_with_fallback(
            "chat_completions_create_stream", model, messages, **kwargs
        )

```

### Supporting Sync, Async, and Streaming Calls

The `_run_with_fallback` helper uses `getattr` to invoke the correct method on each nested provider. This means the same logic covers synchronous `chat_completions_create`, asynchronous `achat_completions_create`, and streaming `chat_completions_create_stream` without duplication. If a provider raises `LLMError`—covering timeouts, rate limits, or model-specific failures—the wrapper silently attempts the next backend in the sequence.

## Registering the Fallback Provider in the Client

### Wiring the Wrapper into `AisuiteClient`

Once instantiated, the fallback provider can be registered directly on the client’s provider map. Any tool or trace that requests the `"fallback"` key will now traverse the chain automatically.

```python
from aisuite.client import AisuiteClient

fallback_keys = ["openai", "anthropic", "gemini"]
fallback_cfgs = [
    {"api_key": "OPENAI_KEY", "model": "gpt-4o"},
    {"api_key": "ANTHROPIC_KEY", "model": "claude-3-sonnet"},
    {"api_key": "GEMINI_KEY", "model": "gemini-1.5-pro"},
]

fallback = FallbackProvider(fallback_keys, fallback_cfgs)

client = AisuiteClient()
client.providers["fallback"] = fallback

```

After registration, selecting the `"fallback"` provider—whether through a `provider_key` argument or direct client access—triggers the ordered chain. If OpenAI is unavailable, aisuite tries Anthropic, then Gemini, raising the last error only when every backend fails.

## Why This Design Enables Resilient Fallbacks

### Uniform Abstract API

Because [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) enforces a single interface across every backend, the `FallbackProvider` never needs provider-specific conditional logic. It forwards `model`, `messages`, and any additional `**kwargs` unchanged, relying on each concrete class to handle its own protocol details.

### Factory-Driven Loading

`ProviderFactory.create_provider` loads modules on demand by mapping keys to filenames under `aisuite/providers/*_provider.py`. Adding a new backend only requires dropping a new provider file into that directory; the fallback wrapper can immediately include it in its configuration list without code changes.

### Centralized Error Handling with `LLMError`

`LLMError` is the base exception class for all LLM-related failures in aisuite. By catching this single exception type, the fallback wrapper treats timeouts, rate limits, or remote errors as generic unavailable signals. You can narrow the catch to specific subclasses if your use case requires finer-grained control.

## Summary

- **aisuite abstracts every backend** behind the `Provider` base class in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py), enabling transparent delegation.
- **`ProviderFactory.create_provider`** dynamically instantiates any provider from a short key and config dict, so the wrapper can build its chain at runtime.
- **The `FallbackProvider` class** stores an ordered list of providers, catches `LLMError`, and tries the next backend until one succeeds.
- **Register the wrapper** on `AisuiteClient.providers` under any key—such as `"fallback"`—to make it available throughout the codebase.
- **Sync, async, and streaming methods** are all supported through a single `_run_with_fallback` helper that uses `getattr` to forward calls.

## Frequently Asked Questions

### What exception should the fallback provider catch?

The wrapper should catch `LLMError`, the base exception defined in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) for LLM-related failures. This covers timeouts, rate limits, and provider-specific errors that signal the backend is unavailable.

### Does aisuite support async and streaming in the fallback chain?

Yes. Because every concrete provider implements `achat_completions_create` and `chat_completions_create_stream`, the wrapper can delegate all three variants through the same `_run_with_fallback` helper, as implemented in the `FallbackProvider` example.

### How does aisuite load providers without hard-coded imports?

`ProviderFactory` in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) uses a naming convention to lazily import the correct module from `aisuite/providers/*_provider.py`. You pass a short string key like `"anthropic"`, and the factory resolves it to the corresponding class automatically.

### Can I use the fallback wrapper in existing aisuite agents?

Yes. After registering the wrapper on `client.providers`, any existing code—including examples such as [`examples/agents/simple_agent.py`](https://github.com/andrewyng/aisuite/blob/main/examples/agents/simple_agent.py)—can reference the fallback chain by its registered key just like a standard provider.