# How to Implement a Custom Provider Adapter for a Non-OpenAI API in FreeLLMAPI > Learn to implement a custom provider adapter for non-OpenAI APIs in FreeLLMAPI. Extend BaseProvider, translate responses, and register your adapter to integrate any LLM. - Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi) - Tags: how-to-guide - Published: 2026-07-01 --- **To implement a custom provider adapter in FreeLLMAPI, extend the `BaseProvider` abstract class, implement the three required methods (`chatCompletion`, `streamChatCompletion`, and `validateKey`), translate the upstream API responses into OpenAI-compatible formats, and register the provider in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts).** FreeLLMAPI (from the `tashfeenahmed/freellmapi` repository) unifies disparate LLM backends behind a single **provider** interface. By implementing a custom adapter, you can integrate any RESTful LLM service—even those with unique request formats or authentication schemes—into the proxy without modifying the core routing or quota-tracking logic. ## Understanding the BaseProvider Contract The foundation of every provider in `tashfeenahmed/freellmapi` is the `BaseProvider` abstract class defined in [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts). This contract enforces three mandatory methods: - **`chatCompletion`**: Handles synchronous, non-streaming requests. - **`streamChatCompletion`**: Handles server-sent event (SSE) streaming responses. - **`validateKey`**: Verifies API key validity against the upstream service. `BaseProvider` also provides utility methods for HTTP handling: `fetchWithTimeout` for aborted requests, `makeId` for generating unique identifiers, and `readSseStream` for parsing SSE streams. Reusing these utilities ensures consistent timeout behavior and error handling across all providers. ## Step-by-Step Implementation ### Create a Subclass of BaseProvider Begin by creating a new TypeScript file in `server/src/providers/`. Your class must extend `BaseProvider` and define the `platform` identifier (used for routing and quota tracking) and a human-readable `name`. ```typescript import { BaseProvider, providerHttpError } from './base.js'; import type { ChatMessage, ChatCompletionResponse, ChatCompletionChunk, CompletionOptions } from '@freellmapi/shared/types.js'; import type { QuotaObservationContext } from '../services/provider-quota.js'; export class MyCoolProvider extends BaseProvider { readonly platform = 'mycool' as const; readonly name = 'My Cool Service'; private readonly baseUrl: string; private readonly extraHeaders: Record; constructor(opts: { baseUrl: string; extraHeaders?: Record }) { super(); this.baseUrl = opts.baseUrl; this.extraHeaders = opts.extraHeaders ?? {}; } // Required methods implemented below... } ``` ### Translate Request and Response Formats The core responsibility of a custom provider adapter is **protocol translation**. In `chatCompletion`, construct the payload expected by your upstream API, then map the response into the OpenAI-compatible `ChatCompletionResponse` shape. ```typescript async chatCompletion( apiKey: string, messages: ChatMessage[], modelId: string, options?: CompletionOptions, quotaContext?: QuotaObservationContext, ): Promise { // Build provider-specific payload const payload = { model: modelId, history: messages, temperature: options?.temperature, maxTokens: options?.max_tokens, }; // Execute with timeout handling const res = await this.fetchWithTimeout(`${this.baseUrl}/chat`, { method: 'POST', headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}`, ...this.extraHeaders, }, body: JSON.stringify(payload), }); // Handle quota observation (optional but recommended) recordQuotaObservationsFromResponse(res, { platform: this.platform, keyId: quotaContext?.keyId, modelId, quotaPoolKey: quotaContext?.quotaPoolKey, endpoint: 'chat/completions', }); // Error handling if (!res.ok) { const err = await res.json().catch(() => ({})); throw providerHttpError(res, `${this.name} API error ${res.status}: ${err.error?.message ?? res.statusText}`); } // Map upstream response to OpenAI format const upstream = await res.json(); return { id: this.makeId(), object: 'chat.completion', created: Math.floor(Date.now() / 1000), model: modelId, choices: [{ index: 0, message: { role: 'assistant', content: upstream.replyText, // Map field names appropriately }, finish_reason: 'stop', }], usage: { prompt_tokens: upstream.usage?.promptTokens ?? 0, completion_tokens: upstream.usage?.completionTokens ?? 0, total_tokens: upstream.usage?.totalTokens ?? 0, }, }; } ``` ### Handle Streaming Responses For streaming support, implement `streamChatCompletion` as an async generator yielding `ChatCompletionChunk` objects. If your upstream API uses SSE, delegate to `this.readSseStream()` from the base class. For non-standard streaming protocols (like Google's Gemini), implement custom parsing logic similar to the pattern found in [`server/src/providers/google.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/google.ts). ```typescript async *streamChatCompletion( apiKey: string, messages: ChatMessage[], modelId: string, options?: CompletionOptions, ): AsyncGenerator { const res = await this.fetchWithTimeout(`${this.baseUrl}/chat/stream`, { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', }, body: JSON.stringify({ model: modelId, messages }), }); if (!res.ok) { throw providerHttpError(res, `${this.name} streaming error ${res.status}`); } // Re-use generic SSE parser if compatible yield* this.readSseStream(res); } ``` ### Implement Key Validation The `validateKey` method performs a lightweight health check. Return `true` for valid credentials, `false` for invalid ones. Most implementations issue a `GET` request to a `/models` or `/status` endpoint and check for 401/403 status codes. ```typescript async validateKey(apiKey: string): Promise { const res = await this.fetchWithTimeout(`${this.baseUrl}/status`, { method: 'GET', headers: { Authorization: `Bearer ${apiKey}` }, }); return res.status !== 401 && res.status !== 403; } ``` ## Registering Your Custom Provider After implementation, expose the provider to the router by registering it in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts): ```typescript import { MyCoolProvider } from './my-cool-provider.js'; register(new MyCoolProvider({ baseUrl: 'https://api.mycoolservice.com/v1', extraHeaders: { 'X-Custom-Header': 'value' }, })); ``` For dynamic, user-supplied endpoints, the repository already contains a resolver pattern. When `platform === 'custom'`, the code constructs a fresh `OpenAICompatProvider` on-the-fly. Replace this with your custom class if the endpoint is not OpenAI-compatible: ```typescript if (platform === 'custom' && baseUrl) { return new MyCoolProvider({ baseUrl: baseUrl.trim() }); } ``` ## Handling Keyless Authentication Some services offer anonymous tiers or use IP-based authentication. To support this, set `this.keyless = true` in your constructor. When this flag is enabled, the proxy layer omits the `Authorization` header from requests to the upstream service. ```typescript constructor(opts: { baseUrl: string }) { super(); this.baseUrl = opts.baseUrl; this.keyless = true; // No API key required } ``` ## Complete Code Example Here is a minimal, production-ready provider file template: ```typescript // server/src/providers/my-cool-provider.ts import type { ChatMessage, ChatCompletionResponse, ChatCompletionChunk, CompletionOptions } from '@freellmapi/shared/types.js'; import { BaseProvider, providerHttpError } from './base.js'; import { recordQuotaObservationsFromResponse, type QuotaObservationContext } from '../services/provider-quota.js'; export class MyCoolProvider extends BaseProvider { readonly platform = 'mycool' as const; readonly name = 'My Cool Service'; private readonly baseUrl: string; constructor(opts: { baseUrl: string }) { super(); this.baseUrl = opts.baseUrl; } async chatCompletion( apiKey: string, messages: ChatMessage[], modelId: string, options?: CompletionOptions, quotaContext?: QuotaObservationContext, ): Promise { const payload = { model: modelId, messages, temperature: options?.temperature }; const res = await this.fetchWithTimeout(`${this.baseUrl}/chat`, { method: 'POST', headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${apiKey}` }, body: JSON.stringify(payload), }); if (!res.ok) throw providerHttpError(res, `${this.name} error`); const data = await res.json(); return { id: this.makeId(), object: 'chat.completion', created: Math.floor(Date.now() / 1000), model: modelId, choices: [{ index: 0, message: { role: 'assistant', content: data.text }, finish_reason: 'stop' }], usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }, }; } async *streamChatCompletion(apiKey: string, messages: ChatMessage[], modelId: string): AsyncGenerator { const res = await this.fetchWithTimeout(`${this.baseUrl}/stream`, { /* ... */ }); yield* this.readSseStream(res); } async validateKey(apiKey: string): Promise { const res = await this.fetchWithTimeout(`${this.baseUrl}/status`, { headers: { Authorization: `Bearer ${apiKey}` } }); return res.ok; } } ``` ## Summary - **Extend `BaseProvider`** from [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts) to create a new adapter. - **Implement three abstract methods**: `chatCompletion`, `streamChatCompletion`, and `validateKey`. - **Translate protocols** by mapping upstream request/response formats to OpenAI-compatible types defined in [`shared/types.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/shared/types.ts). - **Reuse base utilities** like `fetchWithTimeout` and `readSseStream` for consistent HTTP and SSE handling. - **Register the provider** in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) using the `register()` function. - **Set `this.keyless = true`** for services that do not require API key authentication. ## Frequently Asked Questions ### What is the minimum number of methods I must implement in a custom provider? You must implement three abstract methods: `chatCompletion` for synchronous responses, `streamChatCompletion` for streaming (can return an empty generator if unsupported), and `validateKey` for credential verification. The base class provides HTTP utilities like `fetchWithTimeout` and `readSseStream` to simplify these implementations. ### How do I handle APIs that return formats different from OpenAI? Map the upstream response fields to the OpenAI-compatible `ChatCompletionResponse` structure in your `chatCompletion` method. For complex transformations—such as Google's Gemini multimodal responses—refer to [`server/src/providers/google.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/google.ts) as a reference for translating nested or differently named fields into the standard `choices`, `message`, and `usage` schema. ### Can I support both custom static providers and dynamic user-provided endpoints? Yes. Register static providers with `register(new YourProvider())` in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts). For dynamic endpoints, modify the resolver logic in the same file where `platform === 'custom'` to instantiate your provider class with a user-supplied `baseUrl`, similar to how `OpenAICompatProvider` is constructed on-the-fly. ### Where should I place quota tracking logic for my custom provider? Call `recordQuotaObservationsFromResponse()` within your `chatCompletion` method after receiving the upstream response, as shown in the code skeleton. Pass the `res` object along with `platform`, `keyId`, and `modelId` from the `QuotaObservationContext` to enable automatic quota tracking and rate-limiting without manual intervention.