How to Use Async Chat Completions in aisuite: The Complete Guide

Use the acreate coroutine on the Client class to perform non-blocking chat completions, which delegates to each provider's achat_completions_create method for true async I/O or thread-based execution.

The aisuite library by Andrew Ng unifies multiple LLM providers behind a single interface. When building async applications like FastAPI services or Discord bots, blocking the event loop with synchronous HTTP calls destroys performance. This guide explains how to use aisuite's native asynchronous support to keep your application responsive while waiting for model responses.

Understanding the Provider Async Contract

Every provider in aisuite inherits from the abstract Provider base class defined in aisuite/provider.py. This contract specifies the achat_completions_create method, which handles async chat requests.

• Default behavior: The base implementation uses asyncio.to_thread to run the synchronous chat_completions_create method in a background thread. This ensures all providers work asynchronously out-of-the-box without code changes.
• Native optimization: Providers with async SDKs (like OpenAI) override this method to use native await syntax on HTTP requests, avoiding thread overhead and enabling proper connection pooling.

# Conceptual view from aisuite/provider.py

class Provider(ABC):
    async def achat_completions_create(self, messages, **kwargs):
        """Default implementation runs sync version in thread."""
        return await asyncio.to_thread(
            self.chat_completions_create, 
            messages, 
            **kwargs
        )

Native Async Implementation Example

The OpenAI provider in aisuite/providers/openai_provider.py demonstrates a proper native async implementation. Instead of wrapping synchronous code, it instantiates openai.AsyncOpenAI and awaits the native coroutine:

# From aisuite/providers/openai_provider.py

class OpenaiProvider(Provider):
    async def achat_completions_create(self, messages, **kwargs):
        """Native async using OpenAI's AsyncOpenAI client."""
        if not self.async_client:
            self.async_client = openai.AsyncOpenAI(api_key=self.api_key)
        
        response = await self.async_client.chat.completions.create(
            messages=messages,
            **kwargs
        )
        return response

This pattern provides true non-blocking I/O, connection reuse, and proper cancellation support through asyncio.

Using the High-Level Client API

The Client class in aisuite/client.py exposes the acreate method, which resolves the provider from your model string and delegates to the appropriate achat_completions_create implementation.

Method signature:

await client.acreate(
    model="provider:model-name",  # e.g., "openai:gpt-4o-mini"

    messages=[{"role": "user", "content": "Hello"}],
    temperature=0.7,
    max_turns=None,  # Optional tool execution loop limit

    **kwargs
)

The client handles:

• Provider routing: Parses the provider:model format to instantiate the correct backend
• Tool execution loops: When max_turns is provided, acreate manages multi-turn conversations with function calling asynchronously
• Uniform response: Returns a standardized completion object regardless of the underlying provider

Streaming Async Responses

For real-time token generation, use stream=True with acreate. This returns an async iterator that yields completion chunks as they arrive from the provider:

async for chunk in await client.acreate(
    model="openai:gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to 10"}],
    stream=True
):
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Under the hood, providers implement achat_completions_create_stream (or the client detects streaming mode) to provide ChatCompletionChunk objects without buffering the entire response.

Complete Working Examples

Basic Async Completion

import asyncio
from aisuite.client import Client

async def main():
    client = Client()
    
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain async/await in Python"}
    ]
    
    # Non-blocking call to OpenAI

    response = await client.acreate(
        model="openai:gpt-4o-mini",
        messages=messages,
        temperature=0.5
    )
    
    print(response.choices[0].message.content)

if __name__ == "__main__":
    asyncio.run(main())

Concurrent Multi-Provider Calls

async def query_both():
    client = Client()
    
    tasks = [
        client.acreate(
            model="openai:gpt-4o-mini",
            messages=[{"role": "user", "content": "Say hello"}]
        ),
        client.acreate(
            model="anthropic:claude-3-opus-20240229",
            messages=[{"role": "user", "content": "Say hello"}]
        )
    ]
    
    results = await asyncio.gather(*tasks)
    return [r.choices[0].message.content for r in results]

Async with Tool Calling

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {...}
        }
    }
]

response = await client.acreate(
    model="openai:gpt-4o-mini",
    messages=messages,
    tools=tools,
    max_turns=5  # Automatically handle tool execution loops

)

Summary

• aisuite/provider.py defines the achat_completions_create contract with a thread-based default implementation
• aisuite/providers/openai_provider.py showcases native async using openai.AsyncOpenAI
• aisuite/client.py provides the user-facing acreate method that routes to provider implementations
• Use the provider:model format (e.g., "openai:gpt-4o-mini") to specify targets
• Enable stream=True for async token streaming via async iterators
• All providers support async via the base class, but native implementations offer better performance and resource utilization

Frequently Asked Questions

What is the difference between create and acreate in aisuite?

The create method performs synchronous blocking HTTP requests, while acreate is the asynchronous coroutine that yields control back to the event loop during network I/O. Use acreate inside async functions to prevent blocking other concurrent tasks.

Do all providers support native async, or do they use threads?

Not all providers implement native async. The base class in aisuite/provider.py provides a default implementation using asyncio.to_thread, which works for any provider. However, major providers like OpenAI override this with native await calls on their async clients for optimal performance.

How do I handle streaming responses in async mode?

Pass stream=True to client.acreate() and iterate over the result with async for. The response becomes an async iterator yielding ChatCompletionChunk objects. This works identically across providers that support streaming.

Can I use acreate with tools and multi-turn conversations?

Yes. When you provide tools and set max_turns (e.g., max_turns=5), the acreate method in aisuite/client.py automatically manages the conversation loop asynchronously. It will await tool calls and feed results back to the model until completion or until the turn limit is reached, all without blocking your main thread.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}