# Understanding Async Implementation in AISuite: achat_completions and Async Tool Execution

> Explore async implementation in AISuite. Learn how achat_completions and async tool execution leverage asyncio.to_thread for non-blocking LLM operations across providers.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: internals
- Published: 2026-08-04

---

**AISuite delivers a unified asynchronous interface for LLM chat completions and tool execution that automatically bridges synchronous SDKs using `asyncio.to_thread`, ensuring fully non-blocking operations across all providers.**

The `aisuite` library by Andrew Ng provides a standardized, provider-agnostic way to interact with large language models. Understanding the async implementation in aisuite reveals how the framework handles `achat_completions_create` methods and `Tools.aexecute_tool` to support both native asynchronous clients and legacy synchronous code without freezing the event loop.

## The Asynchronous Chat Completion Contract

The foundation of aisuite's async support lies in the abstract `Provider` class defined in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py). This contract mandates that every concrete provider implement two core asynchronous methods:

```python

# aisuite/provider.py

class Provider(ABC):
    @abstractmethod
    async def achat_completions_create(self, model, messages, **kwargs):
        """Return a ChatCompletionResponse."""

    @abstractmethod
    async def achat_completions_create_stream(self, model, messages, **kwargs):
        """Yield ChatCompletionChunk objects."""

```

Concrete providers such as OpenAI implement these using their native async clients. For instance, the OpenAI provider forwards calls directly to `openai.AsyncClient.chat.completions.create` as implemented in [`aisuite/providers/openai_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/openai_provider.py).

## Bridging Synchronous Providers to Async

When a provider only offers a synchronous `completions_create` method, the base `Provider` class supplies a default async implementation. This bridge executes the synchronous code in a background thread using `asyncio.to_thread`, preventing event loop blockage.

The fallback mechanism is validated in [`tests/providers/test_async_provider.py`](https://github.com/andrewyng/aisuite/blob/main/tests/providers/test_async_provider.py):

```python

# tests/providers/test_async_provider.py

async def test_default_async_offloads_sync_to_thread():
    response = await provider.achat_completions_create(...)
    # The sync implementation ran on a separate thread.

```

For streaming, `achat_completions_create_stream` follows an identical pattern. If a provider lacks a native async generator, the base class iterates over the synchronous result and yields each chunk asynchronously, preserving back-pressure semantics.

## Asynchronous Tool Execution

The `Tools` utility class in [`aisuite/utils/tools.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/utils/tools.py) handles execution of user-defined functions that may be either synchronous or asynchronous. The `aexecute_tool` method inspects the callable type and routes accordingly:

```python

# aisuite/utils/tools.py

class Tools:
    @staticmethod
    async def aexecute_tool(tool, *args, **kwargs):
        if inspect.iscoroutinefunction(tool):
            return await tool(*args, **kwargs)       # async path

        else:
            return await asyncio.to_thread(tool, *args, **kwargs)  # sync path

```

**Async tools** are recognized via `inspect.iscoroutinefunction` and awaited directly. **Sync tools** are wrapped in `asyncio.to_thread` to run off the main event loop. This design is tested in [`tests/utils/test_async_tools.py`](https://github.com/andrewyng/aisuite/blob/main/tests/utils/test_async_tools.py), where `test_aexecute_awaits_async_tool` verifies direct awaiting, and `test_aexecute_runs_sync_tool_off_the_loop` confirms that synchronous functions execute in a thread pool without blocking.

## Client Integration and Orchestration

The high-level `Client` class orchestrates LLM calls and tool execution in [`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py). When the LLM returns a tool call request, the client delegates to `Tools.aexecute_tool`:

```python

# aisuite/client.py (excerpt)

if tool_call:
    result = await Tools.aexecute_tool(tool, *args, **kwargs)

```

This integration guarantees that regardless of whether the provider or the tool is inherently synchronous, the external API remains fully asynchronous and awaitable.

## Error Handling and Policy Enforcement

Both the chat-completion bridge and the tool executor respect provider-specific deny policies that restrict certain tool usage. Errors originating from synchronous implementations are captured and re-raised as asynchronous exceptions, preserving full stack traceability. The test `test_aexecute_respects_deny_policy` in [`tests/utils/test_async_tools.py`](https://github.com/andrewyng/aisuite/blob/main/tests/utils/test_async_tools.py) validates that policy enforcement occurs before tool execution begins.

## Practical Implementation Examples

### Calling Async Chat Completions

```python
import asyncio
from aisuite.client import AISuiteClient

async def main():
    client = AISuiteClient()
    response = await client.provider.achat_completions_create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Tell me a joke"}],
    )
    print(response.choices[0].message.content)

asyncio.run(main())

```

### Executing Synchronous Tools

```python
import asyncio
from aisuite.utils.tools import Tools

def fetch_weather(city: str) -> str:
    # Blocking I/O (e.g., requests.get(...))

    return f"The weather in {city} is sunny."

async def demo():
    result = await Tools.aexecute_tool(fetch_weather, "Paris")
    print(result)          # Runs in a thread, does not block the loop.

asyncio.run(demo())

```

### Using Native Async Tools

```python
async def async_fetch_weather(city: str) -> str:
    await asyncio.sleep(0.1)          # Simulate async I/O

    return f"The weather in {city} is rainy."

result = await Tools.aexecute_tool(async_fetch_weather, "Tokyo")
print(result)  # Awaited directly.

```

## Summary

- **AISuite** standardizes asynchronous chat completions through the abstract `Provider` class requiring `achat_completions_create` and `achat_completions_create_stream` implementations.
- **Synchronous providers** are automatically bridged to async using `asyncio.to_thread`, ensuring the event loop never blocks even when underlying SDKs lack native async support.
- **Tool execution** via `Tools.aexecute_tool` automatically detects async functions with `inspect.iscoroutinefunction`, routing sync tools through a thread-pool executor.
- **Key files** include [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) for the contract, [`aisuite/utils/tools.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/utils/tools.py) for execution logic, and corresponding test files validating the thread-offloading behavior.

## Frequently Asked Questions

### How does aisuite handle providers that only offer synchronous APIs?

AISuite automatically bridges synchronous providers by running their blocking methods in a background thread using `asyncio.to_thread`. This fallback is implemented in the base `Provider` class, ensuring that `achat_completions_create` always returns an awaitable object even when the underlying SDK lacks native async support.

### What is the difference between `achat_completions_create` and `achat_completions_create_stream`?

`achat_completions_create` returns a complete `ChatCompletionResponse` object asynchronously, while `achat_completions_create_stream` yields `ChatCompletionChunk` objects asynchronously as they arrive from the provider. Both methods follow the same sync-to-async bridging pattern when providers lack native streaming implementations.

### How does `Tools.aexecute_tool` decide whether to use a thread pool?

The method uses `inspect.iscoroutinefunction(tool)` to detect if the callable is a coroutine. If true, it awaits the tool directly; otherwise, it wraps the call in `asyncio.to_thread` to execute synchronous functions off the main event loop. This ensures responsive async behavior regardless of the tool's implementation.

### Can synchronous tools block the event loop when called through aisuite?

No. The `Tools.aexecute_tool` implementation guarantees that synchronous tools run in a separate thread via `asyncio.to_thread`, preventing event loop blockage. This is verified in the test suite, which confirms that sync tool execution does not interfere with the main event loop's responsiveness.