Architecture of aisuite's Multi-Provider Chat Completion System: A Deep Dive

aisuite implements a plug-in architecture where a single client can call any LLM provider through a common interface, using an abstract Provider base class, a ProviderFactory for dynamic discovery, and concrete provider classes that vendor-specific SDK calls.

The aisuite library—created by Andrew Ng's team—eliminates vendor lock-in by unifying OpenAI, Anthropic, Google, and other providers under one consistent API. Understanding this architecture helps developers extend the library, debug provider issues, or build multi-model applications. This article explores the core components, data flow, and integration patterns based on the actual source code.

---

Core Architectural Components

The system revolves around four main pieces: an abstract base class that defines the contract, concrete implementations per vendor, a factory that discovers and instantiates providers, and a high-level client that orchestrates calls.

The Provider Abstract Base Class

Every vendor implementation inherits from Provider in [aisuite/provider.py](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py#L24-L55). This class establishes the minimal contract that all providers must satisfy:

• chat_completions_create(model, messages, **kwargs) → returns ChatCompletionResponse
• Optional achat_completions_create() for async operations
• Optional chat_completions_create_stream() for streaming responses
• Optional ASR support via asr_transcribe() and asr_translate()

The base class also provides default async implementations using asyncio.to_thread() to offload synchronous SDK calls to a thread pool. Providers with native async APIs—like OpenAI and Anthropic—can override these for true non-blocking I/O.

# aisuite/provider.py - conceptual structure

class Provider(ABC):
    @abstractmethod
    def chat_completions_create(self, model, messages, **kwargs):
        ...
    
    async def achat_completions_create(self, model, messages, **kwargs):
        # Default: thread-offloaded sync call

        return await asyncio.to_thread(
            self.chat_completions_create, model, messages, **kwargs
        )

Concrete Provider Implementations

Each vendor has a dedicated module in aisuite/providers/. These classes translate the generic chat_completions_create call into vendor-specific SDK invocations.

Key providers include:

• OpenaiProvider — wraps openai.OpenAI client
• AnthropicProvider — wraps anthropic.Anthropic client
• GoogleProvider — wraps Google's Gemini SDK
• AzureProvider, AwsbedrockProvider, CohereProvider, etc.

Each provider normalizes the response into the common ChatCompletionResponse format defined in aisuite/framework/.

The ProviderFactory for Dynamic Discovery

The ProviderFactory eliminates hardcoded provider lists. It discovers available providers by scanning for *_provider.py files and maps short keys ("openai", "anthropic") to class names:

1. Converts key to module: "openai" → openai_provider
2. Imports aisuite.providers.{module}
3. Instantiates the {Key}Provider class with user config
4. Returns the ready-to-use provider object

If a key isn't found, ProviderFactory.get_supported_providers() returns the list of available options for helpful error messaging.

from aisuite.provider import ProviderFactory

# Factory creates provider with injected config

provider = ProviderFactory.create_provider(
    "anthropic",
    config={"api_key": "sk-ant-..."}
)

AISuiteClient: The Unified Facade

The AISuiteClient is what applications interact with. Its initialization flow:

1. Accepts a config dict mapping provider keys to credentials
2. Calls ProviderFactory.create_provider() for each entry
3. Stores instances in self.providers: dict[str, Provider]
4. Exposes chat_completions_create(provider_key, ...) that delegates to the selected provider

from aisuite.client import AISuiteClient

client = AISuiteClient({
    "openai": {"api_key": "sk-..."},
    "anthropic": {"api_key": "sk-ant-..."},
    "google": {"api_key": "..."}
})

# Route to any provider with identical call signature

response = client.chat_completions_create(
    provider_key="anthropic",
    model="claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "Explain neural networks"}]
)

---

Data Flow: From Client Call to Provider Response

Understanding how a request traverses the system clarifies where to hook custom logic or debug failures.

User Code
    │
    ▼
AISuiteClient.chat_completions_create("anthropic", ...)
    │
    ▼
self.providers["anthropic"]  (Provider instance from Factory)
    │
    ▼
AnthropicProvider.chat_completions_create(model, messages, **kwargs)
    │
    ▼
anthropic.Anthropic().messages.create(...)  # Vendor SDK

    │
    ▼
Normalize response → ChatCompletionResponse
    │
    ▼
Return to user

Unified data structures in aisuite/framework/ ensure that regardless of provider, you receive:

• ChatCompletionResponse with .choices, .usage, .model
• Message objects with .role, .content, .tool_calls
• ChatCompletionChunk for streaming responses

---

MCP Integration: Cross-Provider Tool Calling

The MCP (Model Context Protocol) integration enables tool use without provider-specific implementation. Key insight: providers don't execute tools directly—they receive tool definitions, return tool calls, and the client handles execution.

The architecture separates concerns:

Layer	Responsibility
Tools class	Wraps MCP tool definitions into Python callables
Provider	Sends tool schemas to LLM, receives tool call requests
MCP client	Executes the actual tool and returns results
AISuiteClient	Orchestrates the loop: LLM call → tool execution → result injection

This means a tool written once works with any provider that supports function calling—no per-vendor tool logic required.

---

Async and Streaming Architecture

The base Provider class provides thread-offloaded async by default, but performance-critical applications benefit from native async overrides.

Default pattern (works for all providers):

# Uses asyncio.to_thread() under the hood

response = await client.achat_completions_create(
    provider_key="google",
    model="gemini-1.5-flash",
    messages=[...]
)

Native async (OpenAI, Anthropic):

These providers override achat_completions_create() to use their SDK's async clients directly, avoiding thread pool overhead.

# aisuite/providers/openai_provider.py - native async example

async def achat_completions_create(self, model, messages, **kwargs):
    response = await self.async_client.chat.completions.create(
        model=model, messages=messages, **kwargs
    )
    return self._normalize_response(response)

Streaming follows the same pattern: base class provides thread-offlined iteration, providers can override for native async generators.

---

Extending the Architecture: Adding a New Provider

The plug-in design makes adding vendors straightforward:

1. Create aisuite/providers/newvendor_provider.py
2. Implement class NewvendorProvider(Provider)
3. Define chat_completions_create() with vendor SDK call
4. Optionally override async/streaming methods
5. Return normalized ChatCompletionResponse

The ProviderFactory automatically discovers the new module—no registration required.

# aisuite/providers/newvendor_provider.py

from aisuite.provider import Provider
from aisuite.framework import ChatCompletionResponse

class NewvendorProvider(Provider):
    def __init__(self, config):
        self.api_key = config["api_key"]
        self.client = NewVendorSDK(api_key=self.api_key)
    
    def chat_completions_create(self, model, messages, **kwargs):
        raw = self.client.generate(model=model, messages=messages)
        return ChatCompletionResponse(
            id=raw.id,
            model=raw.model,
            choices=[...],
            usage=...
        )

---

Summary

• Abstract Provider class in [aisuite/provider.py](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py#L24-L55) defines the unified interface for all vendors
• ProviderFactory dynamically discovers and instantiates providers from *_provider.py modules without hardcoded lists
• Concrete providers in aisuite/providers/ implement vendor-specific SDK calls and normalize responses
• AISuiteClient orchestrates provider loading and delegates chat completion calls transparently
• Common framework in aisuite/framework/ ensures consistent Message, ChatCompletionResponse, and streaming chunk types across all providers
• MCP tool integration separates tool definition from provider implementation, enabling cross-vendor tool use
• Async and streaming support defaults to thread-offloaded execution with native async available for performance-critical providers

---

Frequently Asked Questions

How does aisuite handle different authentication schemes across providers?

Each provider implementation extracts its required credentials from the config dict passed to ProviderFactory.create_provider(). The factory passes the raw config to the provider constructor, and the provider pulls specific keys—api_key, base_url, region, etc.—according to its vendor's requirements. This keeps authentication logic encapsulated per provider rather than centralized.

Can I use aisuite with custom or self-hosted models?

Yes. The architecture supports any provider with an OpenAI-compatible API through the generic provider pattern, or you can implement a custom Provider subclass for proprietary endpoints. The factory-based discovery means your custom provider integrates seamlessly with the same AISuiteClient interface.

What happens if a provider doesn't support streaming or async?

The base Provider class implements these as thread-offloaded fallbacks. For streaming, chat_completions_create_stream() wraps the synchronous call and yields chunks. For async, achat_completions_create() uses asyncio.to_thread(). These defaults work for any provider, though native overrides improve performance where available.

How does tool calling work when providers have different function-calling formats?

The Tools wrapper in [aisuite/mcp/tool_wrapper.py](https://github.com/andrewyng/aisuite/blob/main/aisuite/mcp/tool_wrapper.py) standardizes tool schemas into MCP's universal format before passing to providers. When the LLM returns a tool call, the provider's response normalizer extracts the standardized tool request, and the client executes the actual tool via the MCP layer. This abstracts away vendor-specific function-calling syntax.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}