Understanding Async Implementation in AISuite: achat_completions and Async Tool Execution

AISuite delivers a unified asynchronous interface for LLM chat completions and tool execution that automatically bridges synchronous SDKs using asyncio.to_thread, ensuring fully non-blocking operations across all providers.

The aisuite library by Andrew Ng provides a standardized, provider-agnostic way to interact with large language models. Understanding the async implementation in aisuite reveals how the framework handles achat_completions_create methods and Tools.aexecute_tool to support both native asynchronous clients and legacy synchronous code without freezing the event loop.

The Asynchronous Chat Completion Contract

The foundation of aisuite's async support lies in the abstract Provider class defined in aisuite/provider.py. This contract mandates that every concrete provider implement two core asynchronous methods:

# aisuite/provider.py

class Provider(ABC):
    @abstractmethod
    async def achat_completions_create(self, model, messages, **kwargs):
        """Return a ChatCompletionResponse."""

    @abstractmethod
    async def achat_completions_create_stream(self, model, messages, **kwargs):
        """Yield ChatCompletionChunk objects."""

Concrete providers such as OpenAI implement these using their native async clients. For instance, the OpenAI provider forwards calls directly to openai.AsyncClient.chat.completions.create as implemented in aisuite/providers/openai_provider.py.

Bridging Synchronous Providers to Async

When a provider only offers a synchronous completions_create method, the base Provider class supplies a default async implementation. This bridge executes the synchronous code in a background thread using asyncio.to_thread, preventing event loop blockage.

The fallback mechanism is validated in tests/providers/test_async_provider.py:

# tests/providers/test_async_provider.py

async def test_default_async_offloads_sync_to_thread():
    response = await provider.achat_completions_create(...)
    # The sync implementation ran on a separate thread.

For streaming, achat_completions_create_stream follows an identical pattern. If a provider lacks a native async generator, the base class iterates over the synchronous result and yields each chunk asynchronously, preserving back-pressure semantics.

Asynchronous Tool Execution

The Tools utility class in aisuite/utils/tools.py handles execution of user-defined functions that may be either synchronous or asynchronous. The aexecute_tool method inspects the callable type and routes accordingly:

# aisuite/utils/tools.py

class Tools:
    @staticmethod
    async def aexecute_tool(tool, *args, **kwargs):
        if inspect.iscoroutinefunction(tool):
            return await tool(*args, **kwargs)       # async path

        else:
            return await asyncio.to_thread(tool, *args, **kwargs)  # sync path

Async tools are recognized via inspect.iscoroutinefunction and awaited directly. Sync tools are wrapped in asyncio.to_thread to run off the main event loop. This design is tested in tests/utils/test_async_tools.py, where test_aexecute_awaits_async_tool verifies direct awaiting, and test_aexecute_runs_sync_tool_off_the_loop confirms that synchronous functions execute in a thread pool without blocking.

Client Integration and Orchestration

The high-level Client class orchestrates LLM calls and tool execution in aisuite/client.py. When the LLM returns a tool call request, the client delegates to Tools.aexecute_tool:

# aisuite/client.py (excerpt)

if tool_call:
    result = await Tools.aexecute_tool(tool, *args, **kwargs)

This integration guarantees that regardless of whether the provider or the tool is inherently synchronous, the external API remains fully asynchronous and awaitable.

Error Handling and Policy Enforcement

Both the chat-completion bridge and the tool executor respect provider-specific deny policies that restrict certain tool usage. Errors originating from synchronous implementations are captured and re-raised as asynchronous exceptions, preserving full stack traceability. The test test_aexecute_respects_deny_policy in tests/utils/test_async_tools.py validates that policy enforcement occurs before tool execution begins.

Practical Implementation Examples

Calling Async Chat Completions

import asyncio
from aisuite.client import AISuiteClient

async def main():
    client = AISuiteClient()
    response = await client.provider.achat_completions_create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Tell me a joke"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

Executing Synchronous Tools

import asyncio
from aisuite.utils.tools import Tools

def fetch_weather(city: str) -> str:
    # Blocking I/O (e.g., requests.get(...))

    return f"The weather in {city} is sunny."

async def demo():
    result = await Tools.aexecute_tool(fetch_weather, "Paris")
    print(result)          # Runs in a thread, does not block the loop.

asyncio.run(demo())

Using Native Async Tools

async def async_fetch_weather(city: str) -> str:
    await asyncio.sleep(0.1)          # Simulate async I/O

    return f"The weather in {city} is rainy."

result = await Tools.aexecute_tool(async_fetch_weather, "Tokyo")
print(result)  # Awaited directly.

Summary

• AISuite standardizes asynchronous chat completions through the abstract Provider class requiring achat_completions_create and achat_completions_create_stream implementations.
• Synchronous providers are automatically bridged to async using asyncio.to_thread, ensuring the event loop never blocks even when underlying SDKs lack native async support.
• Tool execution via Tools.aexecute_tool automatically detects async functions with inspect.iscoroutinefunction, routing sync tools through a thread-pool executor.
• Key files include aisuite/provider.py for the contract, aisuite/utils/tools.py for execution logic, and corresponding test files validating the thread-offloading behavior.

Frequently Asked Questions

How does aisuite handle providers that only offer synchronous APIs?

AISuite automatically bridges synchronous providers by running their blocking methods in a background thread using asyncio.to_thread. This fallback is implemented in the base Provider class, ensuring that achat_completions_create always returns an awaitable object even when the underlying SDK lacks native async support.

What is the difference between achat_completions_create and achat_completions_create_stream?

achat_completions_create returns a complete ChatCompletionResponse object asynchronously, while achat_completions_create_stream yields ChatCompletionChunk objects asynchronously as they arrive from the provider. Both methods follow the same sync-to-async bridging pattern when providers lack native streaming implementations.

How does Tools.aexecute_tool decide whether to use a thread pool?

The method uses inspect.iscoroutinefunction(tool) to detect if the callable is a coroutine. If true, it awaits the tool directly; otherwise, it wraps the call in asyncio.to_thread to execute synchronous functions off the main event loop. This ensures responsive async behavior regardless of the tool's implementation.

Can synchronous tools block the event loop when called through aisuite?

No. The Tools.aexecute_tool implementation guarantees that synchronous tools run in a separate thread via asyncio.to_thread, preventing event loop blockage. This is verified in the test suite, which confirms that sync tool execution does not interfere with the main event loop's responsiveness.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}