How to Set Up a Custom Provider for gpt4free: A Complete Implementation Guide

You can set up a custom provider for gpt4free by using ClientFactory.create_client() with your endpoint base_url and optional api_key, which dynamically generates a subclass of OpenaiTemplate to handle any OpenAI-compatible API endpoint.

The gpt4free library (maintained by xtekky) provides a unified interface for interacting with various large language model backends. When you need to connect to a private API or an unsupported service that follows the OpenAI chat-completion schema, you can set up a custom provider without modifying the core library files.

Understanding the Custom Provider Architecture

The custom provider system in gpt4free relies on three core components that work together to translate your configuration into a working client.

The OpenaiTemplate Base Class

OpenaiTemplate is the abstract foundation located in g4f/Provider/template/OpenaiTemplate.py. It implements the complete OpenAI-compatible request/response flow, including model discovery, request building, streaming response handling, and error management. Every custom provider ultimately inherits from this class.

The create_custom_provider Helper

Found in g4f/client/__init__.py (lines 22-68), create_custom_provider is a factory function that dynamically builds a new subclass of OpenaiTemplate. It injects your base_url, optional api_key, supported models list, and other attributes directly into the class definition.

ClientFactory Integration

The ClientFactory class (lines 70-95 in g4f/client/__init__.py) bridges the gap between the generated provider class and the high-level Client or AsyncClient wrappers. When you call ClientFactory.create_client(), it:

1. Invokes create_custom_provider to generate the provider subclass
2. Instantiates a Client instance with that provider
3. Exposes the unified API (ChatCompletion.create, Completion.create, etc.)

Step-by-Step Guide to Setting Up a Custom Provider

Follow these steps to integrate any OpenAI-compatible endpoint into the gpt4free ecosystem.

Prerequisites

Ensure your target API adheres to the OpenAI chat-completion schema. The endpoint must accept POST requests at /v1/chat/completions (or a compatible custom path) with a JSON body containing messages, model, and optional parameters like stream.

Creating a Synchronous Client

Use ClientFactory.create_client() for standard synchronous applications. This method constructs your custom provider and returns a ready-to-use client.

from g4f import ClientFactory

# Configure the custom provider

client = ClientFactory.create_client(
    base_url="https://api.my-custom-llm.com/v1",
    api_key="my-secret-key",
    models=["gpt-4o-mini", "gpt-3.5-turbo"],
    default_model="gpt-4o-mini"
)

# Use the unified API

response = client.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain quantum tunnelling in one sentence."}]
)

print(response)

The factory calls create_custom_provider (defined in g4f/client/__init__.py lines 22-68) to generate a subclass of OpenaiTemplate with your base_url and credentials baked in.

Implementing Asynchronous Streaming

For high-performance applications, use ClientFactory.create_async_client() to enable non-blocking I/O and server-sent event (SSE) streaming.

import asyncio
from g4f import ClientFactory

async def main():
    async_client = ClientFactory.create_async_client(
        base_url="https://my-api.example.com/v1",
        api_key="my-async-key",
        default_model="gpt-3.5-turbo"
    )

    async for chunk in async_client.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku about code."}],
        stream=True
    ):
        if isinstance(chunk, Exception):
            raise chunk
        print(chunk, end="", flush=True)

asyncio.run(main())

The stream=True flag leverages the SSE handling implemented in OpenaiTemplate.create_async_generator (see g4f/Provider/template/OpenaiTemplate.py lines 38-65).

Persisting Your Custom Provider to a Module

If you prefer a named provider class that can be imported and reused across projects, generate the class manually and save it to the g4f/Provider directory.

from g4f.client import create_custom_provider

# Generate the provider class

MyProvider = create_custom_provider(
    base_url="https://api.my-service.com/v1",
    api_key="my-static-key",
    name="MyProvider",
    models=["my-model-1", "my-model-2"]
)

# Persist to a Python module

with open("g4f/Provider/MyProvider.py", "w") as f:
    f.write(f"from ..template.OpenaiTemplate import OpenaiTemplate\n\n"
            f"class MyProvider(OpenaiTemplate):\n"
            f"    base_url = '{MyProvider.base_url}'\n"
            f"    api_key = '{MyProvider.api_key}'\n"
            f"    models = {MyProvider.models}\n")

# Register in the package __init__.py

with open("g4f/Provider/__init__.py", "a") as f:
    f.write("\nfrom .MyProvider import MyProvider")

After registration, you can instantiate the client using the provider name:

from g4f import ClientFactory
client = ClientFactory.create_client(provider="MyProvider")

Key Source Files Reference

Understanding the internal structure helps when debugging or extending functionality.

File	Description
g4f/Provider/template/OpenaiTemplate.py	Implements the full OpenAI-compatible request/response flow including streaming and error handling. All custom providers inherit from this class.
g4f/client/__init__.py	Contains create_custom_provider (lines 22-68) for dynamic class generation and ClientFactory (lines 70-95) for client instantiation.
g4f/Provider/__init__.py	Package entry point where you can register persistent custom provider modules.
etc/tool/create_provider.py	CLI utility that scaffolds providers from raw cURL commands for rapid prototyping.
g4f/Provider/ (directory)	Contains built-in provider implementations (e.g., PollinationsAI.py, DeepInfra.py) that serve as reference implementations of OpenaiTemplate subclasses.

Summary

• Use ClientFactory.create_client() to generate a custom provider on-the-fly by providing base_url, api_key, and supported models.
• The architecture relies on OpenaiTemplate in g4f/Provider/template/OpenaiTemplate.py as the base class, with create_custom_provider in g4f/client/__init__.py handling dynamic subclass generation.
• Both synchronous and asynchronous clients are supported via create_client() and create_async_client(), with full streaming capability through server-sent events.
• For reusable providers, persist the generated class to g4f/Provider/ and register it in __init__.py to enable instantiation by name.

Frequently Asked Questions

What is the minimum required configuration for a custom provider?

The only mandatory parameter is base_url, which must point to your OpenAI-compatible API endpoint. While api_key and models are optional, specifying them ensures proper authentication and model discovery. According to the implementation in g4f/client/__init__.py, the factory will generate a functional provider class with just the base URL, though you may encounter authentication errors if your endpoint requires an API key.

Can I use custom providers with streaming responses?

Yes, custom providers fully support streaming through the stream=True parameter in ChatCompletion.create(). The OpenaiTemplate base class in g4f/Provider/template/OpenaiTemplate.py implements server-sent event (SSE) parsing in its create_async_generator method (lines 38-65), enabling real-time token delivery for both synchronous and asynchronous clients without additional configuration.

How do I debug connection issues with my custom provider?

First, verify that your base_url points to the correct API version path (typically ending in /v1). Check the OpenaiTemplate implementation to ensure your endpoint returns standard OpenAI-compatible JSON structures. If using authentication, confirm that api_key is properly formatted with any required Bearer prefix handled by your endpoint. You can also instantiate the provider class directly and call its methods to isolate whether issues stem from the provider configuration or the client wrapper.

Is it possible to register a custom provider permanently in gpt4free?

Yes, you can persist a custom provider by using create_custom_provider to generate the class, then writing the class definition to a new file in g4f/Provider/ (such as MyProvider.py). You must then append an import statement to g4f/Provider/__init__.py to register it within the package namespace. Once registered, you can instantiate it by name using ClientFactory.create_client(provider="MyProvider") across different projects without repeating the configuration parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}