How to Implement Thread-Safe Provider Initialization in aisuite

Use a per-client threading.RLock with double-checked locking in aisuite/client.py to guard the ProviderFactory.create_provider call, preventing race conditions when multiple threads concurrently initialize the same LLM provider for the first time.

The aisuite library by Andrew Ng enables unified access to multiple LLM providers through a lazy-loading architecture. When building multi-threaded applications with aisuite, concurrent calls to client.chat.completions.create() can trigger a race condition during provider initialization. This article demonstrates how to implement thread-safe provider initialization in aisuite by adding synchronization primitives to the Client class.

The Race Condition in Lazy Provider Initialization

aisuite creates concrete LLM-provider objects lazily—the first time a model is referenced, the Client loads the appropriate provider via ProviderFactory.create_provider. The core initialization logic lives in aisuite/client.py within the create method of the Completions class.

When multiple threads call client.chat.completions.create(...) concurrently, each thread may notice that the requested provider is missing in self.client.providers and attempt to instantiate it simultaneously. This race condition can lead to:

• Duplicate provider objects being created
• Inconsistent state if a provider performs one-time setup (e.g., establishing a session, loading credentials, or warming a cache)
• Unexpected ImportError or ValueError when the module import or initialization is not re-entrant

The repository already flags this issue with a TODO comment in the source code:

# TODO: Add thread-safe provider initialization with lock to prevent race conditions

# when multiple threads try to initialize the same provider simultaneously.

Implementing Double-Checked Locking

The thread-safe fix introduces a per-client threading.RLock stored on the Client instance. Every accessor (chat, audio, etc.) shares the same synchronization primitive. The pattern uses double-checked locking: first check if the provider is present, acquire the lock, re-check, then create the provider if still absent.

This ensures only one thread performs the initialization while others wait or simply reuse the already-created instance. The lock scope is minimal—it protects only the creation block—so contention is negligible after the first initialization.

Code Implementation

Step 1: Add RLock to the Client Class

Modify aisuite/client.py to import threading and initialize the lock in the Client constructor:

# aisuite/client.py

import threading   # ← new import

class Client:
    def __init__(self, provider_configs: dict = {}, extra_param_mode: Literal["strict", "warn", "permissive"] = "warn"):
        # ... existing initialization ...

        self._provider_lock = threading.RLock()   # ← thread-safe lock

        # ...

Step 2: Guard Provider Creation in Completions.create

Update the Completions.create method (around lines 78-88) to use the lock when calling ProviderFactory.create_provider:

# aisuite/client.py – inside Completions.create

if provider_key not in self.client.providers:
    # Double-checked locking

    with self.client._provider_lock:
        if provider_key not in self.client.providers:   # re-check after acquiring lock

            config = self.client.provider_configs.get(provider_key, {})
            self.client.providers[provider_key] = ProviderFactory.create_provider(
                provider_key, config
            )

The surrounding validation logic (lines 66-78) remains unchanged. The lock only wraps the instantiation call to ProviderFactory.create_provider.

Step 3: Multi-Threaded Usage Example

The following example demonstrates safe concurrent access from multiple threads:

import threading
from aisuite import Client

client = Client(provider_configs={"openai": {"api_key": "..."}})

def worker(i):
    response = client.chat.completions.create(
        model="openai:gpt-4",
        messages=[{"role": "user", "content": f"hello from {i}"}],
        max_turns=0,
    )
    print(f"Thread {i}: {response.choices[0].message.content}")

threads = [threading.Thread(target=worker, args=(n,)) for n in range(5)]
for t in threads:
    t.start()
for t in threads:
    t.join()

All five threads safely share the same underlying OpenAI provider instance without race conditions.

Key Files and Architectural Flow

The implementation spans these critical files in the andrewyng/aisuite repository:

• aisuite/provider.py: Defines ProviderFactory and the abstract Provider base class. The create_provider method is the synchronization target.
• aisuite/client.py: Contains the Client class (where the lock is added) and the Completions.create method (lines 66-88) where provider keys are parsed and validated against ProviderFactory.get_supported_providers().
• tests/client/test_manual_tool_calling.py: Unit tests exercising provider creation that validate the thread-safe behavior.
• tests/provider/test_provider.py: Tests the generic ProviderFactory logic.

The architectural flow follows this sequence:

1. User calls client.chat.completions.create(model="openai:gpt-4", ...)
2. Completions.create parses the provider_key from the model string (lines 66-71)
3. Validates the key against ProviderFactory.get_supported_providers() (lines 72-78)
4. Acquires the RLock and checks the providers dictionary again
5. Calls ProviderFactory.create_provider only if the provider is still absent

Summary

• The race condition occurs in aisuite/client.py when multiple threads concurrently trigger lazy provider initialization via ProviderFactory.create_provider.
• Adding a threading.RLock to the Client class provides thread-safe synchronization without requiring global locks or process-wide mutexes.
• Implement double-checked locking to minimize contention and ensure that initialization side-effects (network sessions, credential loading) happen exactly once.
• The lock scope is limited to the initialization block, preserving the library's lazy-loading design and maintaining testability.

Frequently Asked Questions

What causes the race condition in aisuite provider initialization?

The race condition occurs because aisuite uses lazy initialization. When multiple threads call client.chat.completions.create() simultaneously before a provider exists in self.client.providers, each thread attempts to invoke ProviderFactory.create_provider for the same provider key. This results in duplicate object creation and potential inconsistent state if the provider performs non-idempotent setup operations.

Why use RLock instead of a standard Lock in aisuite?

Using threading.RLock (reentrant lock) allows the same thread to acquire the lock multiple times without deadlocking, which provides safer synchronization if the initialization code path involves nested calls that might also try to acquire the lock. For the Client class in aisuite, this prevents potential deadlocks in complex scenarios while still blocking other threads from concurrent initialization.

Does thread-safe initialization affect performance?

No. The lock only guards the provider creation block, which executes exactly once per provider per Client instance. After initialization, subsequent threads simply perform a dictionary lookup on self.client.providers without acquiring the lock, resulting in negligible overhead during normal operation.

Where exactly should the lock be placed in the source code?

Add the threading.RLock instance in aisuite/client.py within the Client.__init__ method as self._provider_lock. Then use it in the Completions.create method around lines 78-88 where ProviderFactory.create_provider is called, following the double-checked locking pattern: check existence, acquire lock, re-check existence, then create.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}