How to Build a Multi-Provider Fallback Mechanism Using aisuite

You can build a multi-provider fallback mechanism using aisuite by implementing a custom FallbackProvider class that wraps the ProviderFactory, iterates over an ordered list of backends, and catches LLMError to automatically delegate to the next available provider.

The andrewyng/aisuite library abstracts LLM and audio backends behind a single Provider interface, which makes it easy to construct a multi-provider fallback mechanism using aisuite without touching provider-specific internals. Every concrete implementation—from OpenAIProvider to AnthropicProvider and GeminiProvider—lives in aisuite/providers/*_provider.py and inherits the same methods defined in aisuite/provider.py.

aisuite Provider Architecture

The Provider Abstract Base Class

In aisuite/provider.py, the Provider class defines the core contract that every backend must implement. Concrete providers override methods such as chat_completions_create, achat_completions_create, and chat_completions_create_stream. Because all providers share this uniform interface, a wrapper can delegate any call without knowing which backend is running underneath.

ProviderFactory Dynamic Loading

The ProviderFactory class in aisuite/provider.py handles dynamic imports using a naming convention. Calling ProviderFactory.create_provider("openai", config) lazily loads the corresponding module and instantiates the class. This factory accepts short string keys—such as "openai", "anthropic", or "gemini"—and a configuration dictionary, then returns a fully initialized provider object.

The Client Provider Registry

The central client in aisuite/client.py maintains a mapping called self.providers that holds active provider instances. When you register a new provider instance under a custom key, the rest of aisuite treats it like any native backend. This registry is what lets you drop a fallback wrapper into the stack transparently.

Implementing the FallbackProvider Wrapper

To create a multi-provider fallback mechanism using aisuite, follow three steps:

1. Define an ordered list of provider keys from primary to secondary.
2. Create a thin wrapper that instantiates each provider via ProviderFactory.
3. Delegate every call through a helper that catches LLMError and retries the next provider.

Wrapper Class Implementation

from aisuite.provider import Provider, ProviderFactory, LLMError
from typing import List, Dict, Any


class FallbackProvider(Provider):
    """Wraps several providers and falls back on error."""
    def __init__(self, provider_keys: List[str], configs: List[Dict[str, Any]]):
        super().__init__()
        if len(provider_keys) != len(configs):
            raise ValueError("Keys and configs must have the same length")
        self._providers: List[Provider] = [
            ProviderFactory.create_provider(k, cfg)
            for k, cfg in zip(provider_keys, configs)
        ]

    def _run_with_fallback(self, method_name: str, *args, **kwargs):
        last_exc = None
        for prov in self._providers:
            try:
                method = getattr(prov, method_name)
                return method(*args, **kwargs)
            except LLMError as exc:
                last_exc = exc
        raise last_exc or LLMError("All providers failed")

    def chat_completions_create(self, model, messages, **kwargs):
        return self._run_with_fallback(
            "chat_completions_create", model, messages, **kwargs
        )

    async def achat_completions_create(self, model, messages, **kwargs):
        return await self._run_with_fallback(
            "achat_completions_create", model, messages, **kwargs
        )

    def chat_completions_create_stream(self, model, messages, **kwargs):
        return self._run_with_fallback(
            "chat_completions_create_stream", model, messages, **kwargs
        )

Supporting Sync, Async, and Streaming Calls

The _run_with_fallback helper uses getattr to invoke the correct method on each nested provider. This means the same logic covers synchronous chat_completions_create, asynchronous achat_completions_create, and streaming chat_completions_create_stream without duplication. If a provider raises LLMError—covering timeouts, rate limits, or model-specific failures—the wrapper silently attempts the next backend in the sequence.

Registering the Fallback Provider in the Client

Wiring the Wrapper into AisuiteClient

Once instantiated, the fallback provider can be registered directly on the client’s provider map. Any tool or trace that requests the "fallback" key will now traverse the chain automatically.

from aisuite.client import AisuiteClient

fallback_keys = ["openai", "anthropic", "gemini"]
fallback_cfgs = [
    {"api_key": "OPENAI_KEY", "model": "gpt-4o"},
    {"api_key": "ANTHROPIC_KEY", "model": "claude-3-sonnet"},
    {"api_key": "GEMINI_KEY", "model": "gemini-1.5-pro"},
]

fallback = FallbackProvider(fallback_keys, fallback_cfgs)

client = AisuiteClient()
client.providers["fallback"] = fallback

After registration, selecting the "fallback" provider—whether through a provider_key argument or direct client access—triggers the ordered chain. If OpenAI is unavailable, aisuite tries Anthropic, then Gemini, raising the last error only when every backend fails.

Why This Design Enables Resilient Fallbacks

Uniform Abstract API

Because aisuite/provider.py enforces a single interface across every backend, the FallbackProvider never needs provider-specific conditional logic. It forwards model, messages, and any additional **kwargs unchanged, relying on each concrete class to handle its own protocol details.

Factory-Driven Loading

ProviderFactory.create_provider loads modules on demand by mapping keys to filenames under aisuite/providers/*_provider.py. Adding a new backend only requires dropping a new provider file into that directory; the fallback wrapper can immediately include it in its configuration list without code changes.

Centralized Error Handling with LLMError

LLMError is the base exception class for all LLM-related failures in aisuite. By catching this single exception type, the fallback wrapper treats timeouts, rate limits, or remote errors as generic unavailable signals. You can narrow the catch to specific subclasses if your use case requires finer-grained control.

Summary

• aisuite abstracts every backend behind the Provider base class in aisuite/provider.py, enabling transparent delegation.
• ProviderFactory.create_provider dynamically instantiates any provider from a short key and config dict, so the wrapper can build its chain at runtime.
• The FallbackProvider class stores an ordered list of providers, catches LLMError, and tries the next backend until one succeeds.
• Register the wrapper on AisuiteClient.providers under any key—such as "fallback"—to make it available throughout the codebase.
• Sync, async, and streaming methods are all supported through a single _run_with_fallback helper that uses getattr to forward calls.

Frequently Asked Questions

What exception should the fallback provider catch?

The wrapper should catch LLMError, the base exception defined in aisuite/provider.py for LLM-related failures. This covers timeouts, rate limits, and provider-specific errors that signal the backend is unavailable.

Does aisuite support async and streaming in the fallback chain?

Yes. Because every concrete provider implements achat_completions_create and chat_completions_create_stream, the wrapper can delegate all three variants through the same _run_with_fallback helper, as implemented in the FallbackProvider example.

How does aisuite load providers without hard-coded imports?

ProviderFactory in aisuite/provider.py uses a naming convention to lazily import the correct module from aisuite/providers/*_provider.py. You pass a short string key like "anthropic", and the factory resolves it to the corresponding class automatically.

Can I use the fallback wrapper in existing aisuite agents?

Yes. After registering the wrapper on client.providers, any existing code—including examples such as examples/agents/simple_agent.py—can reference the fallback chain by its registered key just like a standard provider.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}