# FreeLLMAPI Provider Adapter Architecture: How to Add a New LLM Provider

> Learn the FreeLLMAPI provider adapter architecture. Easily integrate new LLM providers by extending the BaseProvider class and registering your instance. Simplify access to multiple LLM backends.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: architecture
- Published: 2026-06-22

---

**FreeLLMAPI normalizes access to dozens of LLM backends through an abstract `BaseProvider` class that enforces a three-method contract (chatCompletion, streamChatCompletion, validateKey), coupled with a singleton registry pattern that allows new providers to be integrated by extending the base class and registering the instance without modifying the router or business logic.**

FreeLLMAPI (tashfeenahmed/freellmapi) unifies disparate LLM APIs behind a single interface. The **provider adapter architecture** decouples vendor-specific wire protocols from core routing logic, enabling developers to add support for new platforms by implementing a predictable TypeScript interface.

## Provider Adapter Architecture Overview

### The BaseProvider Abstract Class

The foundation of the system is `BaseProvider`, defined in [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts). This abstract class declares three methods that every concrete provider must implement: `chatCompletion`, `streamChatCompletion`, and `validateKey`. It also supplies shared utilities including `fetchWithTimeout` for HTTP requests with global proxy support, `readSseStream` for parsing Server-Sent Events, and `providerHttpError` for standardized error handling.

### Concrete Provider Implementations

Each supported backend extends `BaseProvider` in its own file under `server/src/providers/`. For example, `GoogleProvider` handles the Gemini API in [`server/src/providers/google.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/google.ts), while `OpenAICompatProvider` in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) serves as a generic adapter for any OpenAI-compatible service. These classes translate the universal OpenAI-style schema into vendor-specific request formats.

### The Provider Registry

The registry in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) maintains a singleton `Map<Platform, BaseProvider>` that holds all built-in providers. Helper functions `getProvider`, `resolveProvider`, `hasProvider`, and `getAllProviders` expose this map to the rest of the application. Registration happens via `register(new ProviderClass())`, while the special `custom` platform is resolved dynamically from user-supplied URLs.

### Request Flow

When a request arrives, the router ([`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)) extracts the desired `platform` identifier and calls `getProvider(platform)`. The returned instance executes the requested operation using `proxyFetch` for consistent tracing and proxy configuration. Because all providers adhere to the same abstract interface, higher-level services such as quota tracking and health checks remain agnostic to backend implementation details.

## How to Add a New Provider to FreeLLMAPI

Adding support for a new LLM backend requires five discrete steps. You will create a new TypeScript file, implement three abstract methods, register the class, and update the platform type union.

### Step 1: Create the Provider File

Create a new file under `server/src/providers/` (e.g., [`mycloud.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/mycloud.ts)). Import the base class and types:

```typescript
import { BaseProvider, providerHttpError } from './base.js';
import type { ChatMessage, ChatCompletionResponse, ChatCompletionChunk, QuotaObservationContext } from '@freellmapi/shared/types.js';
import { recordQuotaObservationsFromResponse } from '../services/provider-quota.js';

```

### Step 2: Extend BaseProvider

Define a class that extends `BaseProvider`. Declare the `platform` identifier and `name` as readonly constants. Optionally set `keyless = true` if the service requires no authentication.

```typescript
export class MyCloudProvider extends BaseProvider {
  readonly platform = 'mycloud' as const;
  readonly name = 'MyCloud AI';
  // keyless = false; // set to true if no API key required
}

```

### Step 3: Implement the Required Methods

Implement the three abstract methods: `chatCompletion`, `streamChatCompletion`, and `validateKey`.

**chatCompletion** sends non-streaming requests:

```typescript
async chatCompletion(
  apiKey: string,
  messages: ChatMessage[],
  modelId: string,
  options?: CompletionOptions,
  quotaContext?: QuotaObservationContext,
): Promise<ChatCompletionResponse> {
  const res = await this.fetchWithTimeout('https://api.mycloud.ai/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ model: modelId, messages, ...options }),
  });

  recordQuotaObservationsFromResponse(res, {
    platform: this.platform,
    keyId: quotaContext?.keyId,
    modelId,
    quotaPoolKey: quotaContext?.quotaPoolKey,
    endpoint: 'chat/completions',
  });

  if (!res.ok) {
    const errBody = await res.json().catch(() => ({}));
    throw providerHttpError(res, `MyCloud error ${res.status}: ${errBody?.error?.message ?? res.statusText}`);
  }

  const data = await res.json() as ChatCompletionResponse;
  data._routed_via = { platform: this.platform, model: modelId };
  return data;
}

```

**streamChatCompletion** yields SSE chunks using the base class helper:

```typescript
async *streamChatCompletion(
  apiKey: string,
  messages: ChatMessage[],
  modelId: string,
  options?: CompletionOptions,
  quotaContext?: QuotaObservationContext,
): AsyncGenerator<ChatCompletionChunk> {
  const res = await this.fetchWithTimeout('https://api.mycloud.ai/v1/chat/completions', {
    method: 'POST',
    headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ model: modelId, messages, stream: true, ...options }),
  });

  recordQuotaObservationsFromResponse(res, {
    platform: this.platform,
    keyId: quotaContext?.keyId,
    modelId,
    quotaPoolKey: quotaContext?.quotaPoolKey,
    endpoint: 'chat/completions',
  });
  
  if (!res.ok) {
    const err = await res.json().catch(() => ({}));
    throw providerHttpError(res, `MyCloud streaming error ${res.status}: ${err?.error?.message ?? res.statusText}`);
  }
  
  yield* this.readSseStream(res);
}

```

**validateKey** checks credential validity:

```typescript
async validateKey(apiKey: string, quotaContext?: QuotaObservationContext): Promise<boolean> {
  const res = await this.fetchWithTimeout('https://api.mycloud.ai/v1/models', {
    method: 'GET',
    headers: { Authorization: `Bearer ${apiKey}` },
  });
  // Return false only for confirmed bad credentials (401/403)
  return res.status !== 401 && res.status !== 403;
}

```

### Step 4: Register the Provider

Import and register your class in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts):

```typescript
import { MyCloudProvider } from './mycloud.js';
register(new MyCloudProvider());

```

Once registered, the router automatically recognizes the new platform without code changes.

### Step 5: Update Platform Types

If your platform uses a strict enum rather than string literals, add the identifier to the `Platform` union in [`shared/types.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/shared/types.ts):

```typescript
export type Platform = 'openai' | 'google' | 'mycloud' | 'custom';

```

## Key Files in the Provider System

Understanding the codebase structure helps when debugging or extending adapters:

- **[`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts)** – Defines `BaseProvider` with abstract methods and shared utilities like `fetchWithTimeout` and `readSseStream`.
- **[`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts)** – Singleton registry exposing `getProvider`, `register`, and `resolveProvider`.
- **[`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts)** – Reference implementation for OpenAI-compatible APIs.
- **[`server/src/providers/google.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/google.ts)** – Reference implementation for Gemini, showing non-OpenAI message translation.
- **[`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)** – Entry point that selects providers via the registry.
- **[`shared/types.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/shared/types.ts)** – Contains `Platform` type definitions and shared interfaces.

## Summary

FreeLLMAPI's provider adapter system decouples backend specifics from core routing logic through an abstract class and registry pattern. Key takeaways include:

- **Extend `BaseProvider`** in [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts) to implement `chatCompletion`, `streamChatCompletion`, and `validateKey`.
- **Use shared helpers** like `fetchWithTimeout` and `readSseStream` to ensure consistent timeout, proxy, and SSE handling.
- **Register the instance** in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) using `register(new YourProvider())`.
- **Update `Platform` types** in [`shared/types.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/shared/types.ts) if using strict type unions.
- **Leverage quota tracking** by calling `recordQuotaObservationsFromResponse` in your HTTP handlers.

## Frequently Asked Questions

### Do I need to modify the router to add a new provider?

No. The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) dynamically resolves providers through the registry's `getProvider` function. Once you register your new class in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts), the router automatically recognizes it without additional changes.

### How does streaming work with the BaseProvider?

The `BaseProvider` class provides a `readSseStream` helper that parses Server-Sent Events. In your `streamChatCompletion` implementation, await the HTTP response, validate it, then yield `this.readSseStream(res)` to push chunks to the client while maintaining back-pressure handling.

### Can I create a provider that doesn't require an API key?

Yes. Set the `keyless` property to `true` in your provider class. When `keyless` is enabled, the system skips validation checks and quota observations for that provider, treating it as a public or self-hosted endpoint.

### What error handling pattern should I follow?

Use the `providerHttpError` utility from [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts) to throw standardized errors. Return `false` from `validateKey` only for confirmed authentication failures (HTTP 401/403). For network-level failures, allow exceptions to propagate so health checks mark the key as "inconclusive" rather than definitively bad.