How to Implement a Custom Provider Using AbstractProvider in GPT4Free

To implement a custom provider in GPT4Free, subclass AbstractProvider from g4f/providers/base_provider.py, implement the required create_completion method, declare metadata class attributes like label and working, and import your class in g4f/Provider/__init__.py to auto-register it with the routing system.

The gpt4free library (repository xtekky/gpt4free) routes every LLM request through a provider—a class that encapsulates communication with a specific endpoint. While the framework includes dozens of built-in providers, you can extend it by subclassing AbstractProvider, which lives in g4f/providers/base_provider.py and handles parameter marshaling, streaming helpers, and error bridging.

Understanding the Provider Architecture

All providers inherit from a thin ABC called BaseProvider (defined in g4f/providers/types.py), which establishes the public API surface. AbstractProvider builds on this by implementing the heavy-lifting logic and forcing subclasses to provide a concrete create_completion implementation.

Core Responsibilities

When you subclass AbstractProvider, you must implement:

• create_completion(cls, model, messages, **kwargs): A synchronous method that returns a CreateResult (typically a generator yielding strings or ImageResponse objects). This is called by AbstractProvider.create_function and the async helper create_async defined in the base class.
• Class metadata: Attributes like label, working, supports_stream, and supports_message_history tell the router, CLI (g4f/cli/__init__.py), and GUI whether the provider is available and what capabilities it offers.
• Optional async support: If you inherit from AsyncGeneratorProvider instead, you implement create_async_generator to yield chunks asynchronously.

Model Discovery with ProviderModelMixin

For endpoints that support multiple model names, mix in ProviderModelMixin. This adds get_models(), default_model, and model_aliases attributes, which the router uses in g4f/providers/any_provider.py to match model strings to providers. Many built-in providers use this mixin to expose model discovery without hardcoding lists in multiple places.

Creating a Minimal Synchronous Provider

For a simple HTTP endpoint that returns complete responses (no streaming), subclass AbstractProvider and return a generator yielding the final text.

# g4f/Provider/my_custom_provider.py

from __future__ import annotations

from ..typing import CreateResult, Messages
from ..providers.base_provider import AbstractProvider, ProviderModelMixin

class MyCustomProvider(AbstractProvider, ProviderModelMixin):
    """Wrapper for a fictional MyLLM API endpoint."""
    label = "MyLLM"
    working = True
    supports_stream = False
    supports_message_history = True
    supports_system_message = True

    @classmethod
    def get_models(cls):
        return ["mymodel-v1", "mymodel-v2"]

    @classmethod
    def create_completion(
        cls,
        model: str,
        messages: Messages,
        **kwargs,
    ) -> CreateResult:
        import json, requests

        payload = {"model": model, "messages": messages}
        payload.update(kwargs)

        resp = requests.post(
            "https://api.my-llm.com/v1/chat/completions",
            json=payload,
            headers={"Authorization": f"Bearer {kwargs.get('api_key')}"}
        )
        resp.raise_for_status()
        data = resp.json()

        text = data["choices"][0]["message"]["content"]
        
        def generator():
            yield text
        return generator()

Implementing an Async Streaming Provider

For real-time streaming, inherit from AsyncGeneratorProvider (also in g4f/providers/base_provider.py) and implement create_async_generator.

# g4f/Provider/my_streaming_provider.py

from __future__ import annotations

import json
import aiohttp
from ..typing import AsyncResult, Messages
from ..providers.base_provider import AsyncGeneratorProvider, ProviderModelMixin

class MyStreamingProvider(AsyncGeneratorProvider, ProviderModelMixin):
    label = "MyStreamingLLM"
    working = True
    supports_stream = True
    supports_message_history = True
    supports_system_message = True

    @classmethod
    async def create_async_generator(
        cls,
        model: str,
        messages: Messages,
        stream: bool = True,
        **kwargs,
    ) -> AsyncResult:
        async with aiohttp.ClientSession() as session:
            async with session.post(
                "https://api.my-llm.com/v1/chat/completions",
                json={"model": model, "messages": messages, "stream": stream},
                headers={"Authorization": f"Bearer {kwargs.get('api_key')}"},
            ) as resp:
                async for line in resp.content:
                    if line.startswith(b"data: "):
                        payload = json.loads(line[6:].decode())
                        chunk = payload["choices"][0]["delta"].get("content", "")
                        if chunk:
                            yield chunk

Registering the Provider

The framework auto-discovers providers through a dynamic import scan in g4f/Provider/__init__.py. The file automatically builds Provider.__providers__ by inspecting all imported classes that subclass BaseProvider. Once your module is imported there, it becomes available to the router and CLI.

Add an import statement to g4f/Provider/__init__.py:

from .my_custom_provider import MyCustomProvider
from .my_streaming_provider import MyStreamingProvider

This registration step populates Provider.__map__ (a dictionary mapping class names to providers) and makes your class accessible via g4f.Provider.MyCustomProvider.

Using Your Custom Provider

After registration, use your provider explicitly or through the generic router.

Explicit usage:

import g4f

response = g4f.Provider.MyCustomProvider.create_completion(
    model="mymodel-v1",
    messages=[{"role": "user", "content": "Hello world"}],
    api_key="MY_API_KEY"
)

print("".join(response))

Router usage (requires model aliases setup via ProviderModelMixin):

import g4f

answer = g4f.ChatCompletion.create(
    model="mymodel-v1",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
)
print(answer)

Summary

• Subclass AbstractProvider for synchronous providers or AsyncGeneratorProvider for streaming async providers, both defined in g4f/providers/base_provider.py.
• Implement create_completion (sync) or create_async_generator (async) to return a generator yielding text chunks or image responses.
• Declare metadata attributes (label, working, supports_stream) so the CLI and GUI can filter and display your provider according to the logic in g4f/Provider/__init__.py.
• Mix in ProviderModelMixin if your endpoint supports multiple models and you want automatic model discovery via get_models().
• Import your class in g4f/Provider/__init__.py to register it with the framework's auto-discovery system used by the CLI (g4f/cli/__init__.py) and GUI parsers.

Frequently Asked Questions

What is the difference between BaseProvider and AbstractProvider?

BaseProvider (in g4f/providers/types.py) is a minimal abstract base class defining the interface contract, while AbstractProvider (in g4f/providers/base_provider.py) provides concrete implementations for parameter handling, streaming logic, and error bridging. You almost always want to subclass AbstractProvider rather than BaseProvider directly, as the latter requires you to re-implement framework plumbing.

Do I need to implement both sync and async versions of my provider?

No. If you inherit from AbstractProvider, you only need to implement create_completion. The base class provides create_async automatically by running your sync code in an executor. For high-performance streaming, inherit from AsyncGeneratorProvider and implement create_async_generator instead, which the router will detect via the supports_stream attribute.

How does the framework discover my new provider?

The g4f/Provider/__init__.py file dynamically scans all imported modules and collects any class that is a subclass of BaseProvider into Provider.__providers__. As long as you import your class in that file (or in a file already imported there), it will appear in the CLI --provider list and the GUI dropdown automatically, as both interfaces query the __providers__ list and filter by the working attribute.

Why is my provider not showing up in the CLI or GUI?

Ensure your class has working = True and that you have imported the module in g4f/Provider/__init__.py. The CLI (g4f/cli/__init__.py) and GUI parsers filter providers using the working attribute; setting it to False hides the provider from user interfaces while keeping it available for direct programmatic use.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}