How to Implement a Custom LLM Provider Adapter in AI-Suite

To implement a custom LLM provider adapter in AI-Suite, create a Python module in aisuite/providers/<name>_provider.py that subclasses the Provider abstract base class and implements the chat_completions_create method to translate between AI-Suite's message format and your vendor's API.

AI-Suite by Andrew Ng provides a unified interface for multiple large language model (LLM) vendors through a consistent abstraction layer. When you need to integrate a new commercial API or self-hosted endpoint, you must implement a custom LLM provider adapter that conforms to the framework's standardized interface. This guide walks through the architecture, required methods, and implementation patterns based on the actual source code in the andrewyng/aisuite repository.

Understanding the Core Architecture

AI-Suite abstracts every LLM behind a small, well-defined interface composed of several key components.

The Provider Abstract Base Class

Located in aisuite/provider.py, the abstract Provider class declares the methods every adapter must implement. At minimum, providers must override chat_completions_create for synchronous chat completion calls. The base class also provides optional hooks for async variants (achat_completions_create), streaming (chat_completions_create_stream), and audio transcription support through an Audio subclass.

Important: The ProviderInterface located in aisuite/framework/provider_interface.py is obsolete and maintained only for backward compatibility. All new custom adapters must subclass Provider instead.

ProviderFactory Auto-Discovery

The ProviderFactory (also in aisuite/provider.py) handles automatic loading and instantiation. It scans the aisuite/providers/ directory for files ending with _provider.py, dynamically imports them, and instantiates classes named <Name>Provider. For example, a file named myvendor_provider.py must contain a class named MyvendorProvider.

Message Converters and Audio Support

Message converters handle translation between AI-Suite's framework-wide Message objects and vendor-specific payload shapes. Reference implementations include OpenAICompliantMessageConverter in aisuite/providers/openai_provider.py and AnthropicMessageConverter in aisuite/providers/anthropic_provider.py. For audio capabilities, providers implement an Audio subclass (e.g., OpenAIAudio, HuggingfaceAudio) exposed via self.audio in the provider's __init__ method.

Step-by-Step Implementation Guide

Follow these steps to create a fully functional custom adapter.

1. Create the Provider Module

Create a new file at aisuite/providers/<myvendor>_provider.py. The filename must end with _provider.py because ProviderFactory scans for that specific pattern.

2. Define the Provider Class

Subclass Provider and initialize your vendor's SDK client:

from aisuite.provider import Provider, LLMError

class MyvendorProvider(Provider):
    def __init__(self, **config):
        # Initialize the vendor-specific SDK

        self.client = MyVendorSDK(**config)
        
        # Optional: Initialize audio support

        super().__init__()
        self.audio = MyvendorAudio(self.client)
        
        # Optional: Set up message converter

        self.converter = MyvendorMessageConverter()

The class name must be the capitalized provider key plus "Provider" (e.g., MyvendorProvider for provider key "myvendor").

3. Implement Synchronous Chat Completion

Override chat_completions_create to handle the core API call:

def chat_completions_create(self, model, messages, **kwargs):
    try:
        # Convert framework messages to vendor payload

        payload = self.converter.convert_request(messages)
        
        # Call vendor API

        raw = self.client.chat_completion(
            model=model, messages=payload, **kwargs
        )
        
        # Convert vendor response to AI-Suite format

        return self.converter.convert_response(raw)
    except Exception as exc:
        raise LLMError(f"Myvendor error: {exc}") from exc

4. Add Async Support (Optional)

If the vendor SDK supports async operations, override achat_completions_create. If omitted, the base Provider class automatically runs the synchronous method in a thread pool:

async def achat_completions_create(self, model, messages, **kwargs):
    # Implementation for truly async SDKs

    payload = self.converter.convert_request(messages)
    raw = await self.client.achat_completion(model=model, messages=payload, **kwargs)
    return self.converter.convert_response(raw)

5. Implement Streaming (Optional)

For real-time streaming support, implement chat_completions_create_stream yielding ChatCompletionChunk objects:

def chat_completions_create_stream(self, model, messages, **kwargs):
    payload = self.converter.convert_request(messages)
    for chunk in self.client.stream_chat(model=model, messages=payload, **kwargs):
        yield ChatCompletionChunk(
            choices=[ChunkChoice(delta=Message(role="assistant", content=chunk["text"]))]
        )

6. Add Audio Support (Optional)

Create an Audio subclass for transcription capabilities:

from aisuite.provider import Audio
from aisuite.framework.message import TranscriptionResult

class MyvendorAudio(Audio):
    class Transcriptions(Audio.Transcription):
        def __init__(self, client):
            self.client = client
            
        def create(self, model, file, **kwargs) -> TranscriptionResult:
            # Call vendor transcription endpoint

            result = self.client.transcribe(model=model, file=file, **kwargs)
            return TranscriptionResult(text=result["text"])

Expose it via self.audio in the provider's __init__ method.

Complete Code Example

Here is a minimal, runnable skeleton demonstrating all required components:

# aisuite/providers/myvendor_provider.py

from aisuite.provider import Provider, LLMError, Audio
from aisuite.framework.message import Message, TranscriptionResult
from aisuite.framework import ChatCompletionResponse

class MyvendorMessageConverter:
    def convert_request(self, messages):
        """Convert framework Message objects to vendor format."""
        return [{"role": m.role, "content": m.content or ""} for m in messages]
    
    def convert_response(self, resp):
        """Build ChatCompletionResponse from vendor response."""
        result = ChatCompletionResponse()
        result.choices[0].message = Message(
            role="assistant", content=resp["text"]
        )
        return result

class MyvendorProvider(Provider):
    def __init__(self, **config):
        self.client = MyVendorSDK(**config)
        super().__init__()
        self.converter = MyvendorMessageConverter()
    
    def chat_completions_create(self, model, messages, **kwargs):
        try:
            payload = self.converter.convert_request(messages)
            raw = self.client.chat_completion(
                model=model, messages=payload, **kwargs
            )
            return self.converter.convert_response(raw)
        except Exception as exc:
            raise LLMError(f"Myvendor error: {exc}") from exc

# Usage in application code

from aisuite.provider import ProviderFactory

config = {"myvendor": {"api_key": "sk-...", "endpoint": "https://api.myvendor.com"}}
provider = ProviderFactory.create_provider("myvendor", config["myvendor"])

response = provider.chat_completions_create(
    model="my-model",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

Wiring the Adapter into the Framework

No explicit registration is required. When users configure your provider:

provider: myvendor
myvendor:
  api_key: "MY-KEY"
  endpoint: "https://api.myvendor.com"

The framework calls ProviderFactory.create_provider("myvendor", config["myvendor"]), which builds the module path aisuite.providers.myvendor_provider and loads the MyvendorProvider class automatically. As long as the naming convention is respected, the new adapter is instantly usable alongside built-in providers like OpenAI and Anthropic.

Summary

• Create a file named <provider>_provider.py in aisuite/providers/ following the strict naming convention required by ProviderFactory.
• Subclass Provider from aisuite/provider.py and name the class <Name>Provider to match your provider key.
• Implement chat_completions_create to handle message conversion via a converter class and execute API calls through your vendor's SDK.
• Optional features include async support (achat_completions_create), streaming (chat_completions_create_stream), and audio transcription by subclassing Audio.
• The obsolete ProviderInterface in aisuite/framework/provider_interface.py should be ignored; always inherit from Provider for new implementations.

Frequently Asked Questions

What is the difference between Provider and ProviderInterface?

ProviderInterface in aisuite/framework/provider_interface.py is a legacy interface maintained solely for backward compatibility. All new custom LLM provider adapters must subclass Provider from aisuite/provider.py, which provides enhanced capabilities including built-in async fallback support, standardized audio handling, and better error management through LLMError.

Do I need to register my custom adapter anywhere in the codebase?

No. AI-Suite uses automatic discovery via ProviderFactory. As long as your file follows the <name>_provider.py naming convention in the aisuite/providers/ directory and your class is named <Name>Provider, the framework loads it automatically when referenced in user configuration without requiring manual registration or imports.

How do I handle different message formats between AI-Suite and my vendor's API?

Implement a dedicated converter class following the pattern used by AnthropicMessageConverter or OpenAICompliantMessageConverter. This class should implement convert_request to transform AI-Suite Message objects into your vendor's payload format, and convert_response to map the vendor's response back to ChatCompletionResponse objects defined in aisuite/framework/message.py.

Can I add support for real-time streaming responses?

Yes. Override chat_completions_create_stream and optionally achat_completions_create_stream to yield ChatCompletionChunk instances as defined in aisuite/framework/chat_completion_chunk.py. Each yielded chunk should contain delta updates that the AI-Suite client aggregates into the final response. If you do not implement these methods, the base Provider class falls back to running your synchronous method in an executor thread.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}