How to Implement chat_completions_create for a New LLM Provider in AISuite

To implement chat_completions_create for a new LLM provider in AISuite, extend the ProviderInterface class in Python to handle HTTP translation, create a corresponding TypeScript Provider implementation for the JavaScript SDK, and register both in the provider index files.

Adding support for a new large language model to AISuite requires implementing the standardized chat completion interface to abstract provider-specific API calls. This guide demonstrates the complete implementation process using the actual source code from the andrewyng/aisuite repository, covering both the Python backend and JavaScript SDK requirements.

Understanding the Provider Abstraction

AISuite unifies LLM interactions through two core interfaces. In Python, the legacy ProviderInterface in aisuite/framework/provider_interface.py defines the chat_completion_create method that translates generic requests into provider-specific HTTP calls. The modern JavaScript SDK expects classes implementing the Provider interface from aisuite-js/src/core/base-provider.ts, which declares chatCompletion for synchronous requests and streamChatCompletion for streaming responses.

Step-by-Step Implementation

1. Create the Python Provider Class

Create a new Python file in aisuite/providers/ that inherits from ProviderInterface and implements the chat_completion_create method. This method must accept messages, model, and optional temperature parameters, translate them into the provider's native payload format, and return the raw JSON response.

# aisuite/providers/myllm.py

from .provider_interface import ProviderInterface
import requests

class MyLLMProvider(ProviderInterface):
    """Concrete implementation for MyLLM."""

    def __init__(self, api_key: str, endpoint: str = "https://api.myllm.com/v1/chat"):
        self.api_key = api_key
        self.endpoint = endpoint

    def chat_completion_create(self, messages=None, model=None, temperature=0):
        payload = {
            "model": model,
            "messages": messages,
            "temperature": temperature,
        }
        headers = {
            "Authorization": f"Bearer {self.api_key}",
            "Content-Type": "application/json"
        }
        resp = requests.post(
            self.endpoint,
            json=payload,
            headers=headers,
            timeout=30
        )
        resp.raise_for_status()
        return resp.json()

2. Implement the TypeScript Wrapper

Create a TypeScript provider in aisuite-js/src/providers/myllm/provider.ts that satisfies the Provider interface. The wrapper typically forwards calls to the Python implementation via AISuite's RPC layer or directly invokes the REST endpoint.

// aisuite-js/src/providers/myllm/provider.ts
import { Provider } from '../../core/base-provider';
import {
  ChatCompletionRequest,
  ChatCompletionResponse,
  ChatCompletionChunk
} from '../../types';
import { client } from '../../client';

export class MyLLMProvider implements Provider {
  readonly name = 'myllm';

  async chatCompletion(
    request: ChatCompletionRequest,
    options?: RequestOptions
  ): Promise<ChatCompletionResponse> {
    return await client.chat_completion_create(
      request.messages,
      request.model,
      request.temperature
    );
  }

  async *streamChatCompletion(
    request: ChatCompletionRequest,
    options?: RequestOptions
  ): AsyncIterable<ChatCompletionChunk> {
    const resp = await this.chatCompletion(request, options);
    yield* resp.choices.map(c => ({
      ...c,
      finish_reason: 'stop'
    } as ChatCompletionChunk));
  }
}

3. Register the Provider

Export the new TypeScript provider in aisuite-js/src/providers/index.ts to make it discoverable by the SDK.

// aisuite-js/src/providers/index.ts
export * from './openai';
export * from './mistral';
export * from './groq';
export * from './anthropic';
export * from './myllm';

For Python registration, add the class to aisuite/providers/__init__.py if your version uses explicit exports.

4. Update Configuration Schema

Add provider-specific configuration keys to aisuite/mcp/config.py and update .env.sample to support credentials like API keys and custom endpoints. This allows users to configure the provider through environment variables or configuration files.

5. Write Tests

Verify that your implementation returns correctly shaped ChatCompletionResponse objects. Test both the Python chat_completion_create output and the TypeScript wrapper's chatCompletion method to ensure they conform to the type definitions in aisuite-js/src/types/chat.ts.

Reference Implementations

Study existing providers to ensure your implementation follows AISuite conventions:

• OpenAI: See aisuite-js/src/providers/openai/provider.ts for a complete reference implementation
• Mistral: See aisuite-js/src/providers/mistral/provider.ts for alternative patterns

Both demonstrate proper error handling, request formatting, and response parsing consistent with the Provider interface contract.

Usage Example

After implementation, use the new provider through the AISuite client:

import { AISuiteClient } from 'aisuite-js';
import { MyLLMProvider } from 'aisuite-js/providers/myllm';

const client = new AISuiteClient({
  provider: new MyLLMProvider(),
});

const response = await client.chatCompletion({
  model: 'my-llm-v1',
  messages: [{ role: 'user', content: 'Hello, world!' }],
});
console.log(response.choices[0].message.content);

Summary

• Extend ProviderInterface in aisuite/framework/provider_interface.py and implement chat_completion_create to handle provider-specific HTTP translation in Python.
• Implement the Provider interface in TypeScript at aisuite-js/src/core/base-provider.ts, providing chatCompletion and streamChatCompletion methods.
• Register exports in aisuite-js/src/providers/index.ts (and aisuite/providers/__init__.py for Python) to enable discovery.
• Update configuration in aisuite/mcp/config.py to support provider credentials.
• Reference existing providers like OpenAI and Mistral for implementation patterns that ensure compatibility with AISuite's chat completion workflow.

Frequently Asked Questions

What parameters must the Python chat_completion_create method accept?

The chat_completion_create method in aisuite/framework/provider_interface.py requires messages (list of message objects), model (string identifier), and optionally temperature (float). Your implementation should translate these into the provider's native API format and return a JSON response conforming to the ChatCompletionResponse shape defined in aisuite-js/src/types/chat.ts.

Do I need to implement both Python and JavaScript to add a new provider?

Yes. AISuite requires a Python class implementing chat_completion_create for the backend logic, and a TypeScript class implementing the Provider interface for the JavaScript SDK. The TypeScript wrapper typically forwards requests to the Python implementation via AISuite's RPC bridge, or directly calls the provider's REST endpoint while maintaining the interface contract.

How do I handle streaming responses in my implementation?

For streaming support, implement the streamChatCompletion generator method in your TypeScript provider to yield ChatCompletionChunk objects. If your underlying LLM supports streaming, translate its SSE (Server-Sent Events) or chunked responses into AISuite's chunk format. If not, fall back to wrapping the complete chatCompletion response as a single chunk with finish_reason: 'stop'.

Where should I add API key configuration for my new provider?

Add provider-specific configuration entries to aisuite/mcp/config.py for Python-side credential loading, and update the .env.sample file to document the required environment variables. This ensures users can configure your provider using the same pattern as existing providers like OpenAI or Anthropic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}