How to Add New LLM Providers to FreeLLMAPI: A Complete Implementation Guide

To add a new LLM provider to FreeLLMAPI, implement a class extending BaseProvider (or reuse OpenAICompatProvider for OpenAI-compatible endpoints) and register it in server/src/providers/index.ts using the register() function.

FreeLLMAPI is a modular, open-source gateway that normalizes access to multiple large language model providers through a unified interface. Adding support for a new provider—whether it follows the OpenAI API specification or uses a custom protocol like Google Gemini—requires only a few lines of code in the provider registry. The system automatically handles routing, health checks, and quota tracking for any provider implementing the BaseProvider contract.

The Provider Architecture

FreeLLMAPI uses a hierarchical provider system defined in server/src/providers/base.ts. The BaseProvider abstract class defines the contract that all providers must implement, including methods for chat completions, streaming, and key validation.

For services that follow the OpenAI API specification, you can reuse OpenAICompatProvider located in server/src/providers/openai-compat.ts. This class handles request formatting, authentication headers, and tool-call normalization automatically. For non-standard protocols, you must create a custom class—see GoogleProvider in server/src/providers/google.ts as the reference implementation for custom request handling.

Step-by-Step: Adding an OpenAI-Compatible Provider

Most modern LLM providers (Groq, OpenRouter, etc.) use OpenAI-compatible endpoints. Adding one requires a single registration call in server/src/providers/index.ts.

1. Register the Provider

Add a register() call to server/src/providers/index.ts that instantiates OpenAICompatProvider with your provider's configuration:

// Add to server/src/providers/index.ts
register(new OpenAICompatProvider({
  platform: 'mynewprovider',          // Unique identifier used in API requests
  name: 'MyNewProvider',              // Human-readable display name
  baseUrl: 'https://api.mynewprovider.com/v1', // Provider's OpenAI-compatible base URL
  extraHeaders: {
    'X-My-Header': 'value',          // Optional: static headers required by provider
  },
  timeoutMs: 20000,                   // Optional: override default 15s timeout
  keyless: false,                     // Set true for free anonymous tiers
}));

This pattern mirrors the existing Groq registration block found in the same file.

2. Verify System Integration

No changes are required in server/src/services/router.ts. The router automatically resolves your new provider using getProvider(platform) or resolveProvider(platform, baseUrl) based on the registration above. Similarly, the health-check endpoint in server/src/services/health.ts automatically validates API keys for providers where keyless is false.

Configuration Options for OpenAI-Compatible Providers

The OpenAICompatProvider constructor accepts several options to handle provider-specific quirks without custom code:

keyless: For providers offering free tiers without authentication, set keyless: true. This causes the authHeader() method to return an empty object (see lines 90‑92 of server/src/providers/openai-compat.ts), preventing the Authorization header from being sent.

register(new OpenAICompatProvider({
  platform: 'examplefree',
  name: 'Example Free AI',
  baseUrl: 'https://api.examplefree.ai/v1',
  keyless: true,
}));

timeoutMs: Override the default 15-second timeout for slow backends. The timeout is applied in the fetchWithTimeout call (lines 101‑108 of server/src/providers/openai-compat.ts).

register(new OpenAICompatProvider({
  platform: 'slowmodel',
  name: 'SlowModel AI',
  baseUrl: 'https://slowmodel.ai/api/v1',
  timeoutMs: 120000,  // 2 minutes
}));

forceSingleToolCall: For providers that do not support parallel tool calls, set forceSingleToolCall: true. This forces parallel_tool_calls to false when tools are present (see resolveParallelToolCalls() at lines 53‑60 of server/src/providers/openai-compat.ts).

register(new OpenAICompatProvider({
  platform: 'singletool',
  name: 'SingleTool AI',
  baseUrl: 'https://singletool.ai/v1',
  forceSingleToolCall: true,
}));

Implementing Custom Protocol Providers

When a service does not follow the OpenAI schema (e.g., Google Gemini), you must implement a dedicated class extending BaseProvider. The GoogleProvider in server/src/providers/google.ts demonstrates the required pattern:

// server/src/providers/google.ts (simplified)
export class GoogleProvider extends BaseProvider {
  readonly platform = 'google';
  readonly name = 'Google Gemini';
  
  // Implement abstract methods:
  // chatCompletion(), streamChatCompletion(), validateKey()
  // Plus any provider-specific logic (e.g., thoughtSignature caching)
}

After creating the class, register it in server/src/providers/index.ts:

register(new GoogleProvider());

Provider Registration and System Integration

Once registered in server/src/providers/index.ts, providers are automatically exposed throughout the system:

• Routing: server/src/services/router.ts uses getProvider(platform) to resolve the correct implementation for incoming requests.
• Health Checks: server/src/services/health.ts automatically validates provider API keys using the validateKey() method (unless keyless is enabled).
• Quota Tracking: Rate-limiting and model-grouping logic operate on the common BaseProvider interface, requiring no additional configuration for new providers.

Summary

• Extend BaseProvider located in server/src/providers/base.ts for custom protocols, or reuse OpenAICompatProvider in server/src/providers/openai-compat.ts for OpenAI-compatible endpoints.
• Register the provider in server/src/providers/index.ts using the register() function with a unique platform identifier and appropriate options.
• Configure provider-specific behavior via constructor options including baseUrl, extraHeaders, timeoutMs, keyless, and forceSingleToolCall.
• Automatic integration occurs through server/src/services/router.ts for request routing and server/src/services/health.ts for key validation.

Frequently Asked Questions

Do I need to modify the router to add a new provider?

No. The router in server/src/services/router.ts dynamically resolves providers using getProvider(platform) and resolveProvider(platform, baseUrl). Once you register a provider in server/src/providers/index.ts, the router automatically handles requests targeting that platform identifier without requiring code changes.

How do I add a provider that does not require an API key?

Set the keyless: true option in the OpenAICompatProvider constructor. This flag causes the authHeader() method to return an empty object, ensuring the system does not send an Authorization header. This is useful for providers offering free anonymous tiers or public endpoints.

Can I customize the request timeout for a specific provider?

Yes. Pass the timeoutMs option when constructing OpenAICompatProvider. The value is passed to fetchWithTimeout in the provider's request logic. For example, timeoutMs: 120000 sets a 2-minute timeout for slow model endpoints, overriding the default 15-second limit.

What if my provider only supports single tool calls and not parallel function calling?

Use the forceSingleToolCall: true option when registering an OpenAICompatProvider. This setting forces the parallel_tool_calls parameter to false when tools are present in the request, accommodating providers that do not support concurrent function invocations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}