What Is the Difference Between AsyncProvider and AsyncGeneratorProvider in gpt4free?

AsyncProvider returns a single complete string after processing, while AsyncGeneratorProvider yields a stream of partial chunks during generation.

The gpt4free library (xtekky/gpt4free) abstracts AI model interactions through provider classes defined in g4f/providers/base_provider.py. When building asynchronous providers, developers must choose between two base classes: AsyncProvider and AsyncGeneratorProvider. Understanding the difference between AsyncProvider and AsyncGeneratorProvider is essential for implementing correct streaming or non-streaming behavior.

Core Architectural Differences

Execution Model: Single Response vs. Streaming

The fundamental distinction lies in how each class handles the model's output. AsyncProvider is designed for single-shot completions. It aggregates the entire model response into one string and returns it only after generation completes. This matches APIs that return complete JSON responses without streaming.

Conversely, AsyncGeneratorProvider implements a streaming execution model. It yields partial results—typically individual tokens or chunks—as soon as they are generated. This allows real-time display of AI responses and reduces perceived latency for long generations.

Abstract Method Signatures

Each class enforces a specific interface through abstract methods defined in g4f/providers/base_provider.py.

AsyncProvider (lines 33‑84) requires:

@staticmethod
async def create_async(model: str, messages: Messages, **kwargs) -> str:
    ...

This method must return a complete str after awaiting the model's response.

AsyncGeneratorProvider (lines 84‑122) requires:

@staticmethod
async def create_async_generator(model: str, messages: Messages, **kwargs) -> AsyncResult:
    ...

Here, AsyncResult is an asynchronous generator that yields string chunks. The method uses yield instead of return, allowing the caller to iterate over partial results.

Implementation Details in gpt4free

AsyncProvider: Non-Streaming Asynchronous Calls

In g4f/providers/base_provider.py, AsyncProvider implements a synchronous façade through create_completion. This method calls asyncio.run() on create_async, blocking until the full string is available:

def create_completion(cls, model, messages, **kwargs):
    return asyncio.run(cls.create_async(model, messages, **kwargs))

Because the entire response is buffered, AsyncProvider does not set supports_stream = True. Timeout handling applies to the entire operation via AbstractProvider.async_create_function, which wraps the call in asyncio.wait_for.

AsyncGeneratorProvider: Real-Time Token Streaming

AsyncGeneratorProvider handles streaming through an asynchronous generator pattern. The create_async_generator method yields chunks as they arrive from the model API. To support synchronous consumption, the class provides create_completion using to_sync_generator:

def create_completion(cls, model, messages, **kwargs):
    return to_sync_generator(
        cls.create_async_generator(model, messages, **kwargs)
    )

This helper, defined in g4f/providers/asyncio.py, bridges async generators to synchronous iterators.

AsyncGeneratorProvider explicitly sets supports_stream = True and use_stream_timeout = True. According to the source code in base_provider.py (lines 84‑122), this enables per‑chunk timeout handling through a custom async_create_function that applies stream_timeout or timeout to each iteration of the generator.

Code Examples

Implementing a Simple AsyncProvider

The following example shows a minimal provider that returns a complete response after an async delay. This matches the pattern found in g4f/providers/base_provider.py for AsyncProvider:

import asyncio
from g4f.providers.base_provider import AsyncProvider, Messages

class EchoAsyncProvider(AsyncProvider):
    """Echoes back the last user message after a short async pause."""

    @staticmethod
    async def create_async(model: str, messages: Messages, **kwargs) -> str:
        # Simulate async I/O (e.g., an HTTP request)

        await asyncio.sleep(0.1)
        # Return the content of the last message

        return messages[-1]["content"]

Calling the provider synchronously:

result = EchoAsyncProvider.create_completion(
    model="any-model",
    messages=[{"role": "user", "content": "Hello"}],
)
print(result)  # → "Hello"

Implementing a Streaming AsyncGeneratorProvider

This example demonstrates streaming behavior by yielding numbered chunks. It follows the AsyncGeneratorProvider pattern from g4f/providers/base_provider.py (lines 84‑122):

import asyncio
from g4f.providers.base_provider import AsyncGeneratorProvider, Messages, AsyncResult

class CountAsyncGeneratorProvider(AsyncGeneratorProvider):
    """Yields numbers 1‑5 as strings, one per async iteration."""

    @staticmethod
    async def create_async_generator(model: str, messages: Messages, **kwargs) -> AsyncResult:
        for i in range(1, 6):
            await asyncio.sleep(0.2)  # simulate per‑token latency

            yield str(i)              # each chunk is yielded immediately

Consuming the stream synchronously:

for chunk in CountAsyncGeneratorProvider.create_completion(
    model="any-model",
    messages=[{"role": "user", "content": "Count"}],
):
    print(chunk, end=" ")   # → 1 2 3 4 5

Both examples inherit helper methods such as get_parameters from AbstractProvider, but they differ in what they must implement (create_async vs create_async_generator) and how the framework handles their output (single string vs async iterator).

Summary

• AsyncProvider is designed for single‑shot completions, requiring implementation of create_async to return a complete str, and uses asyncio.run for its synchronous façade.
• AsyncGeneratorProvider enables real‑time streaming through create_async_generator, yields partial chunks via an async generator, and converts to sync iteration using to_sync_generator.
• Both classes reside in g4f/providers/base_provider.py and inherit common timeout and parameter handling from AbstractProvider, but only AsyncGeneratorProvider sets supports_stream = True for per‑chunk timeout handling.

Frequently Asked Questions

Can I convert an AsyncProvider to support streaming?

No, AsyncProvider is architecturally designed for single‑response completion. To support streaming, you must subclass AsyncGeneratorProvider instead and implement create_async_generator to yield chunks as they arrive. The framework distinguishes between these two patterns through the supports_stream class attribute, which is only True for AsyncGeneratorProvider.

How does timeout handling differ between the two provider types?

AsyncProvider applies timeouts to the entire operation using asyncio.wait_for within AbstractProvider.async_create_function, failing if the whole response takes too long. AsyncGeneratorProvider implements custom timeout logic that can apply to each individual chunk via stream_timeout or timeout parameters, allowing the stream to continue as long as each token arrives within the time limit.

Which provider type should I use for Server‑Sent Events (SSE) APIs?

Use AsyncGeneratorProvider for SSE or any API that returns partial results over time. Implement create_async_generator to parse the SSE stream and yield individual data chunks. This aligns with the streaming architecture in g4f/providers/base_provider.py and ensures proper integration with the framework's streaming timeout and synchronous iteration helpers.

Do both providers support synchronous calling patterns?

Yes, both abstract classes provide a synchronous create_completion method, but they handle the conversion differently. AsyncProvider uses asyncio.run to execute create_async and return the final string. AsyncGeneratorProvider uses to_sync_generator from g4f/providers/asyncio.py to convert the async generator into a synchronous iterator that yields chunks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}