# FreeLLMAPI Provider Adapter Architecture: A Technical Deep Dive

> Explore the FreeLLMAPI provider adapter architecture. Learn how it unifies LLM services with a BaseProvider class for consistent interfaces and flexible provider-specific formats.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: deep-dive
- Published: 2026-08-27

---

**The FreeLLMAPI provider adapter architecture isolates every LLM service behind a unified abstraction layer, using an abstract `BaseProvider` class to enforce consistent interfaces for chat completions, streaming, and API key validation while allowing concrete adapters to handle provider-specific wire formats.**

FreeLLMAPI implements a provider adapter architecture that decouples the core server logic from individual LLM service implementations. This design pattern ensures that the router and service layers interact with any supported platform—whether Google Gemini, Groq, or OpenAI-compatible endpoints—through a single, well-typed contract. By centralizing shared concerns like timeout handling, error parsing, and streaming safety in a base class, the system remains extensible without requiring changes to the routing logic when adding new providers.

## Core Components of the Provider Adapter Architecture

The architecture consists of three distinct layers that separate interface definition from implementation details.

### BaseProvider Abstract Class

The foundation of the provider adapter architecture resides in [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts), where the `BaseProvider` abstract class defines the common contract that every adapter must fulfill.

```typescript
export abstract class BaseProvider {
  abstract readonly platform: Platform;
  abstract readonly name: string;
  keyless = false;                     // true for free‑tier gateways

  abstract chatCompletion(
    apiKey: string,
    messages: ChatMessage[],
    modelId: string,
    options?: CompletionOptions,
    quotaContext?: QuotaObservationContext,
  ): Promise<ChatCompletionResponse>;

  abstract streamChatCompletion(
    apiKey: string,
    messages: ChatMessage[],
    modelId: string,
    options?: CompletionOptions,
    quotaContext?: QuotaObservationContext,
  ): AsyncGenerator<ChatCompletionChunk>;

  abstract validateKey(apiKey: string, quotaContext?: QuotaObservationContext): Promise<KeyValidationResult>;

  // ── shared helpers (timeout, error parsing, stall watchdog, etc.) ──
}

```

All concrete adapters inherit shared utilities from this base class, including `fetchWithTimeout` for consistent timeout behavior, `providerHttpError` for back-off parsing, and `readWithStallTimeout` for streaming safety. This guarantees uniform error handling and resilience characteristics across every LLM integration.

### Concrete Provider Adapters

Each LLM service extends `BaseProvider` to implement the three abstract methods using provider-specific wire formats. The `GoogleProvider` in [`server/src/providers/google.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/google.ts) demonstrates this pattern for the Gemini API:

```typescript
export class GoogleProvider extends BaseProvider {
  readonly platform = 'google' as const;
  readonly name = 'Google Gemini';

  async chatCompletion(...) { /* build Gemini request, call fetchWithTimeout */ }
  async *streamChatCompletion(...) { /* read SSE, apply readWithStallTimeout */ }
  async validateKey(...) { /* probe /v1/models or /v1/health */ }
}

```

This file contains Gemini-specific logic such as signature caching and system-prompt folding while reusing the generic helpers from `BaseProvider`. Other implementations like `OpenAICompatProvider` ([`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts)), `CohereProvider`, and `CloudflareProvider` follow identical structural patterns, differing only in endpoint URLs, request shaping, and provider-specific quirks.

### Provider Registry

The central registry in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) maintains a singleton `Map<Platform, BaseProvider>` that stores instantiated adapters. Registration occurs at module load time:

```typescript
const providers = new Map<Platform, BaseProvider>();

function register(provider: BaseProvider) {
  providers.set(provider.platform, provider);
}

// Example registration
register(new GoogleProvider({ timeoutMs: 60_000 }));
register(new OpenAICompatProvider({ platform: 'groq', name: 'Groq', baseUrl: 'https://api.groq.com/openai/v1' }));

```

The router retrieves adapters using `providers.get(platform)`, ensuring the service layer never contains provider-specific conditional logic.

## How the Architecture Processes Requests

When handling a chat completion request, the router resolves the appropriate adapter from the registry and delegates execution:

```typescript
import { providers } from '@/server/src/providers/index.js';

// Resolve the adapter
const provider = providers.get(request.platform);
if (!provider) throw new Error('Unsupported platform');

// Perform a non‑streaming completion
const response = await provider.chatCompletion(
  apiKey,
  messages,
  modelId,
  { temperature: 0.7, max_tokens: 512 },
);

```

The same abstraction applies to streaming responses via `streamChatCompletion()` and API key validation through `validateKey()`. The returned `ChatCompletionResponse` remains uniform regardless of whether the underlying provider uses OpenAI-compatible JSON or Gemini's proprietary format.

## Extending the Provider Adapter Architecture

Adding support for a new LLM service requires only two steps:

1. **Implement the contract**: Create a class extending `BaseProvider` in a new file under `server/src/providers/`, implementing the three abstract methods with the provider's specific HTTP logic.
2. **Register the instance**: Import the class into [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) and call `register(new MyNewProvider())` to add it to the singleton map.

This modular approach allows developers to introduce new platforms without modifying the router, validation middleware, or quota tracking systems.

## Summary

- **Abstract Contract**: The `BaseProvider` class in [`server/src/providers/base.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/base.ts) defines the mandatory interface (`chatCompletion`, `streamChatCompletion`, `validateKey`) and shared utilities for timeout and error handling.
- **Concrete Implementations**: Provider-specific adapters like `GoogleProvider` and `OpenAICompatProvider` extend the base class to handle unique wire formats while inheriting common resilience patterns.
- **Central Registry**: A `Map<Platform, BaseProvider>` in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) manages singleton instances and enables runtime lookup by platform identifier.
- **Router Isolation**: The service layer retrieves adapters via `providers.get(platform)`, ensuring complete decoupling from provider-specific implementation details.
- **Extensibility**: New LLM services require only a new adapter class and a single registration call, with no changes needed to the core routing logic.

## Frequently Asked Questions

### What is the role of BaseProvider in FreeLLMAPI?

`BaseProvider` serves as the abstract contract that enforces uniformity across all LLM integrations. It defines three required methods—`chatCompletion`, `streamChatCompletion`, and `validateKey`—while providing inherited helpers like `fetchWithTimeout` and `readWithStallTimeout` to ensure consistent timeout behavior and error handling across every provider.

### How does the provider registry manage adapter instances?

The registry in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) maintains a singleton `Map<Platform, BaseProvider>` that stores one instance of each concrete adapter. The `register()` function populates this map at module load time, mapping platform identifiers (like `'google'` or `'groq'`) to their respective adapter instances for runtime resolution.

### What methods must a concrete provider implement?

Every concrete adapter must implement three abstract methods defined in `BaseProvider`: `chatCompletion()` for non-streaming requests, `streamChatCompletion()` for streaming responses (returning an `AsyncGenerator`), and `validateKey()` for API key verification. These implementations handle provider-specific HTTP calls while returning standardized response types.

### How does the architecture handle provider-specific errors?

The `BaseProvider` class includes shared error parsing utilities such as `providerHttpError` that concrete adapters invoke when processing HTTP responses. This centralizes back-off logic and error classification, allowing the router to handle failures uniformly regardless of whether the underlying provider returns OpenAI-style error objects or Gemini-specific status codes.