# Architecture of aisuite's Multi-Provider Chat Completion System: A Deep Dive

> Explore the architecture of aisuite's multi-provider chat completion system. Discover how a plug-in design enables seamless integration with various LLM providers.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: architecture
- Published: 2026-08-03

---

**aisuite implements a plug-in architecture where a single client can call any LLM provider through a common interface, using an abstract `Provider` base class, a `ProviderFactory` for dynamic discovery, and concrete provider classes that vendor-specific SDK calls.**

The [aisuite](https://github.com/andrewyng/aisuite) library—created by Andrew Ng's team—eliminates vendor lock-in by unifying OpenAI, Anthropic, Google, and other providers under one consistent API. Understanding this architecture helps developers extend the library, debug provider issues, or build multi-model applications. This article explores the core components, data flow, and integration patterns based on the actual source code.

---

## Core Architectural Components

The system revolves around four main pieces: an abstract base class that defines the contract, concrete implementations per vendor, a factory that discovers and instantiates providers, and a high-level client that orchestrates calls.

### The `Provider` Abstract Base Class

Every vendor implementation inherits from `Provider` in [[`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py)](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py#L24-L55). This class establishes the **minimal contract** that all providers must satisfy:

- `chat_completions_create(model, messages, **kwargs)` → returns `ChatCompletionResponse`
- Optional `achat_completions_create()` for async operations
- Optional `chat_completions_create_stream()` for streaming responses
- Optional ASR support via `asr_transcribe()` and `asr_translate()`

The base class also provides **default async implementations** using `asyncio.to_thread()` to offload synchronous SDK calls to a thread pool. Providers with native async APIs—like OpenAI and Anthropic—can override these for true non-blocking I/O.

```python

# aisuite/provider.py - conceptual structure

class Provider(ABC):
    @abstractmethod
    def chat_completions_create(self, model, messages, **kwargs):
        ...
    
    async def achat_completions_create(self, model, messages, **kwargs):
        # Default: thread-offloaded sync call

        return await asyncio.to_thread(
            self.chat_completions_create, model, messages, **kwargs
        )

```

### Concrete Provider Implementations

Each vendor has a dedicated module in `aisuite/providers/`. These classes translate the generic `chat_completions_create` call into vendor-specific SDK invocations.

**Key providers include:**

- **`OpenaiProvider`** — wraps `openai.OpenAI` client
- **`AnthropicProvider`** — wraps `anthropic.Anthropic` client  
- **`GoogleProvider`** — wraps Google's Gemini SDK
- **`AzureProvider`**, **`AwsbedrockProvider`**, **`CohereProvider`**, etc.

Each provider normalizes the response into the common `ChatCompletionResponse` format defined in [`aisuite/framework/`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/).

### The `ProviderFactory` for Dynamic Discovery

The [**`ProviderFactory`**](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py#L92-L112) eliminates hardcoded provider lists. It discovers available providers by scanning for `*_provider.py` files and maps short keys (`"openai"`, `"anthropic"`) to class names:

1. Converts key to module: `"openai"` → `openai_provider`
2. Imports `aisuite.providers.{module}`
3. Instantiates the `{Key}Provider` class with user config
4. Returns the ready-to-use provider object

If a key isn't found, `ProviderFactory.get_supported_providers()` returns the list of available options for helpful error messaging.

```python
from aisuite.provider import ProviderFactory

# Factory creates provider with injected config

provider = ProviderFactory.create_provider(
    "anthropic",
    config={"api_key": "sk-ant-..."}
)

```

### `AISuiteClient`: The Unified Facade

The [`AISuiteClient`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py#L28-L84) is what applications interact with. Its initialization flow:

1. Accepts a config dict mapping provider keys to credentials
2. Calls `ProviderFactory.create_provider()` for each entry
3. Stores instances in `self.providers: dict[str, Provider]`
4. Exposes `chat_completions_create(provider_key, ...)` that delegates to the selected provider

```python
from aisuite.client import AISuiteClient

client = AISuiteClient({
    "openai": {"api_key": "sk-..."},
    "anthropic": {"api_key": "sk-ant-..."},
    "google": {"api_key": "..."}
})

# Route to any provider with identical call signature

response = client.chat_completions_create(
    provider_key="anthropic",
    model="claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "Explain neural networks"}]
)

```

---

## Data Flow: From Client Call to Provider Response

Understanding how a request traverses the system clarifies where to hook custom logic or debug failures.

```

User Code
    │
    ▼
AISuiteClient.chat_completions_create("anthropic", ...)
    │
    ▼
self.providers["anthropic"]  (Provider instance from Factory)
    │
    ▼
AnthropicProvider.chat_completions_create(model, messages, **kwargs)
    │
    ▼
anthropic.Anthropic().messages.create(...)  # Vendor SDK

    │
    ▼
Normalize response → ChatCompletionResponse
    │
    ▼
Return to user

```

**Unified data structures** in [`aisuite/framework/`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/) ensure that regardless of provider, you receive:

- `ChatCompletionResponse` with `.choices`, `.usage`, `.model`
- `Message` objects with `.role`, `.content`, `.tool_calls`
- `ChatCompletionChunk` for streaming responses

---

## MCP Integration: Cross-Provider Tool Calling

The [**MCP (Model Context Protocol)**](https://github.com/andrewyng/aisuite/blob/main/aisuite/mcp/tool_wrapper.py) integration enables tool use without provider-specific implementation. Key insight: **providers don't execute tools directly**—they receive tool definitions, return tool calls, and the client handles execution.

The architecture separates concerns:

| Layer | Responsibility |
|-------|---------------|
| `Tools` class | Wraps MCP tool definitions into Python callables |
| Provider | Sends tool schemas to LLM, receives tool call requests |
| MCP client | Executes the actual tool and returns results |
| `AISuiteClient` | Orchestrates the loop: LLM call → tool execution → result injection |

This means a tool written once works with **any** provider that supports function calling—no per-vendor tool logic required.

---

## Async and Streaming Architecture

The base `Provider` class provides **thread-offloaded async** by default, but performance-critical applications benefit from **native async overrides**.

**Default pattern (works for all providers):**

```python

# Uses asyncio.to_thread() under the hood

response = await client.achat_completions_create(
    provider_key="google",
    model="gemini-1.5-flash",
    messages=[...]
)

```

**Native async (OpenAI, Anthropic):**

These providers override `achat_completions_create()` to use their SDK's async clients directly, avoiding thread pool overhead.

```python

# aisuite/providers/openai_provider.py - native async example

async def achat_completions_create(self, model, messages, **kwargs):
    response = await self.async_client.chat.completions.create(
        model=model, messages=messages, **kwargs
    )
    return self._normalize_response(response)

```

**Streaming** follows the same pattern: base class provides thread-offlined iteration, providers can override for native async generators.

---

## Extending the Architecture: Adding a New Provider

The plug-in design makes adding vendors straightforward:

1. Create [`aisuite/providers/newvendor_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/newvendor_provider.py)
2. Implement `class NewvendorProvider(Provider)`
3. Define `chat_completions_create()` with vendor SDK call
4. Optionally override async/streaming methods
5. Return normalized `ChatCompletionResponse`

The `ProviderFactory` automatically discovers the new module—no registration required.

```python

# aisuite/providers/newvendor_provider.py

from aisuite.provider import Provider
from aisuite.framework import ChatCompletionResponse

class NewvendorProvider(Provider):
    def __init__(self, config):
        self.api_key = config["api_key"]
        self.client = NewVendorSDK(api_key=self.api_key)
    
    def chat_completions_create(self, model, messages, **kwargs):
        raw = self.client.generate(model=model, messages=messages)
        return ChatCompletionResponse(
            id=raw.id,
            model=raw.model,
            choices=[...],
            usage=...
        )

```

---

## Summary

- **Abstract `Provider` class** in [[`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py)](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py#L24-L55) defines the unified interface for all vendors
- **`ProviderFactory`** dynamically discovers and instantiates providers from `*_provider.py` modules without hardcoded lists
- **Concrete providers** in `aisuite/providers/` implement vendor-specific SDK calls and normalize responses
- **`AISuiteClient`** orchestrates provider loading and delegates chat completion calls transparently
- **Common framework** in `aisuite/framework/` ensures consistent `Message`, `ChatCompletionResponse`, and streaming chunk types across all providers
- **MCP tool integration** separates tool definition from provider implementation, enabling cross-vendor tool use
- **Async and streaming** support defaults to thread-offloaded execution with native async available for performance-critical providers

---

## Frequently Asked Questions

### How does aisuite handle different authentication schemes across providers?

Each provider implementation extracts its required credentials from the config dict passed to `ProviderFactory.create_provider()`. The factory passes the raw config to the provider constructor, and the provider pulls specific keys—`api_key`, `base_url`, `region`, etc.—according to its vendor's requirements. This keeps authentication logic encapsulated per provider rather than centralized.

### Can I use aisuite with custom or self-hosted models?

Yes. The architecture supports any provider with an OpenAI-compatible API through the generic provider pattern, or you can implement a custom `Provider` subclass for proprietary endpoints. The factory-based discovery means your custom provider integrates seamlessly with the same `AISuiteClient` interface.

### What happens if a provider doesn't support streaming or async?

The base `Provider` class implements these as thread-offloaded fallbacks. For streaming, `chat_completions_create_stream()` wraps the synchronous call and yields chunks. For async, `achat_completions_create()` uses `asyncio.to_thread()`. These defaults work for any provider, though native overrides improve performance where available.

### How does tool calling work when providers have different function-calling formats?

The `Tools` wrapper in [[`aisuite/mcp/tool_wrapper.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/mcp/tool_wrapper.py)](https://github.com/andrewyng/aisuite/blob/main/aisuite/mcp/tool_wrapper.py) standardizes tool schemas into MCP's universal format before passing to providers. When the LLM returns a tool call, the provider's response normalizer extracts the standardized tool request, and the client executes the actual tool via the MCP layer. This abstracts away vendor-specific function-calling syntax.