# How to Add a New Provider to FreeLLMAPI: A Step-by-Step Implementation Guide

> Learn how to add a new provider to FreeLLMAPI with this step-by-step guide. Implement required methods and register your provider for seamless integration.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-27

---

**To add a new provider to FreeLLMAPI, create a class extending `BaseProvider` (or reuse `OpenAICompatProvider` for OpenAI-compatible APIs), implement the three required methods (`chatCompletion`, `streamChatCompletion`, and `validateKey`), and register the instance in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) using the `register()` function.**

FreeLLMAPI routes every LLM request through a centralized provider registry that maps platform identifiers to provider implementations. Adding a new provider to the `tashfeenahmed/freellmapi` repository requires implementing the abstract contract defined in the base class and exposing your implementation to the router. This guide covers both the quick path for OpenAI-compatible endpoints and the full custom implementation for proprietary APIs.

## Understanding the Provider Architecture

FreeLLMAPI uses a registry pattern centered on [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts). This file maintains a singleton `providers` map that connects platform strings (like `"openai"` or `"anthropic"`) to provider instances.

Each provider must implement the `BaseProvider` abstract class from [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts). The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) retrieves the appropriate provider using `getProvider(platform)` and delegates requests to three core methods:

- **`chatCompletion`** – Handles synchronous completion requests
- **`streamChatCompletion`** – Handles Server-Sent Events (SSE) streaming responses
- **`validateKey`** – Validates API keys before routing requests

The `BaseProvider` class supplies essential utilities including `fetchWithTimeout`, `readSseStream`, and `validationResult`, so you only need to implement the HTTP call logic specific to your endpoint.

## Method 1: Adding an OpenAI-Compatible Provider

If your target API follows the OpenAI request/response format, use the existing `OpenAICompatProvider`. This generic implementation already handles the three abstract methods; you only need to configure the base URL and metadata.

Create a new file or add to an existing provider file:

```typescript
// server/src/providers/myai.ts
import { OpenAICompatProvider } from './openai-compat.js';
import { register } from './index.js';

register(new OpenAICompatProvider({
  platform: 'myai',
  name: 'MyAI',
  baseUrl: 'https://api.myai.com/v1',
  timeoutMs: 90_000,           // Optional: increase timeout for cold starts
  extraHeaders: { 'X-Custom-Header': 'free-llmapi' }
}));

```

This approach works for most modern LLM APIs that mimic the OpenAI `/v1/chat/completions` endpoint structure.

## Method 2: Creating a Custom Provider

For APIs with non-standard formats, extend `BaseProvider` directly in [`server/src/providers/myai.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/myai.ts):

```typescript
import { BaseProvider, providerHttpError } from './base.js';
import type { 
  ChatMessage, 
  ChatCompletionResponse, 
  CompletionOptions 
} from '@freellmapi/shared/types.js';

export class MyAIProvider extends BaseProvider {
  readonly platform = 'myai' as const;
  readonly name = 'MyAI';

  async chatCompletion(
    apiKey: string, 
    messages: ChatMessage[], 
    modelId: string, 
    options?: CompletionOptions
  ): Promise<ChatCompletionResponse> {
    const res = await this.fetchWithTimeout(
      'https://api.myai.com/v1/chat/completions',
      {
        method: 'POST',
        headers: { 
          'Authorization': `Bearer ${apiKey}`, 
          'Content-Type': 'application/json' 
        },
        body: JSON.stringify({ model: modelId, messages, ...options }),
      }
    );
    
    if (!res.ok) {
      throw providerHttpError(res, 'MyAI chat failed', await res.json());
    }
    
    return res.json() as Promise<ChatCompletionResponse>;
  }

  async *streamChatCompletion(
    apiKey: string, 
    messages: ChatMessage[], 
    modelId: string, 
    options?: CompletionOptions
  ) {
    const res = await this.fetchWithTimeout(
      'https://api.myai.com/v1/chat/completions',
      {
        method: 'POST',
        headers: { 
          'Authorization': `Bearer ${apiKey}`, 
          'Content-Type': 'application/json' 
        },
        body: JSON.stringify({ 
          model: modelId, 
          messages, 
          stream: true, 
          ...options 
        }),
      }
    );
    
    if (!res.ok) {
      throw providerHttpError(res, 'MyAI stream failed', await res.json());
    }
    
    // Re-use the SSE helper from BaseProvider
    yield* this.readSseStream(res);
  }

  async validateKey(apiKey: string): Promise<boolean> {
    const res = await this.fetchWithTimeout(
      'https://api.myai.com/v1/models',
      { headers: { Authorization: `Bearer ${apiKey}` } }
    );
    return this.validationResult(res);
  }
}

```

The `fetchWithTimeout` method handles request timeouts and retries. The `readSseStream` method parses Server-Sent Events into the standard chunk format. Use `providerHttpError` to standardize error responses across the application.

## Registering Your Provider

After creating your provider class, register it in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts):

```typescript
// server/src/providers/index.ts
import { MyAIProvider } from './myai.js';

// ... existing registrations ...
register(new MyAIProvider());

```

The `register` function inserts the instance into the internal `providers` map, making it available to `getProvider()` calls throughout the application. The `platform` property you defined in your class serves as the lookup key.

## Configuration and Customization

Pass an options object to the constructor to customize behavior:

```typescript
register(new MyAIProvider({ 
  timeoutMs: 120_000,  // Default is typically 60s
  extraHeaders: { 'X-Provider-Source': 'freellmapi' }
}));

```

Available configuration options include:

- **`timeoutMs`** – Request timeout in milliseconds
- **`extraHeaders`** – Headers appended to every request
- **`retryAttempts`** – Number of automatic retries for failed requests

These options are processed by the `BaseProvider` constructor and applied automatically to all HTTP requests made by your provider.

## Testing Your Implementation

Validate your provider integration using the existing test suite:

```bash
pnpm test

# or

npm test

```

Add unit tests under `server/src/services/__tests__/` to verify edge cases specific to your provider. Ensure your `validateKey` method correctly handles both valid and invalid API keys, and verify that streaming responses properly yield chunks in the expected format.

## Summary

- **Extend `BaseProvider`** or reuse **`OpenAICompatProvider`** depending on API compatibility
- **Implement three methods**: `chatCompletion`, `streamChatCompletion`, and `validateKey`
- **Register the instance** in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) using the `register()` function
- **Configure timeouts** and headers via the constructor options object
- **Use built-in helpers** like `fetchWithTimeout` and `readSseStream` to handle networking complexity
- **The router** automatically discovers your provider via the `platform` identifier without requiring modifications to [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)

## Frequently Asked Questions

### Do I need to modify the router to add a new provider?

No. The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) dynamically retrieves providers using `getProvider(platform)`. As long as you register your provider in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts), the router automatically discovers it without code changes.

### Can I add a provider that only supports non-streaming responses?

Yes. Implement `chatCompletion` normally for synchronous responses. For `streamChatCompletion`, you can either throw a `Method not implemented` error or aggregate the full response into a single chunk. The base class handles both patterns gracefully.

### How do I handle API rate limits and retries?

The `BaseProvider` class automatically parses `Retry-After` headers and implements exponential backoff. You can customize retry behavior by passing `retryAttempts` in the constructor options. For provider-specific rate limit handling, override the `handleError` method in your subclass.

### What is the difference between `platform` and `name` properties?

The `platform` property is a unique string identifier used internally for routing (e.g., `"myai"`), while the `name` property is a human-readable display name shown in logs and user interfaces (e.g., `"MyAI Enterprise"`). The `platform` must be unique across all registered providers.