FreeLLMAPI Provider Adapter Architecture: A Technical Deep Dive

The FreeLLMAPI provider adapter architecture isolates every LLM service behind a unified abstraction layer, using an abstract BaseProvider class to enforce consistent interfaces for chat completions, streaming, and API key validation while allowing concrete adapters to handle provider-specific wire formats.

FreeLLMAPI implements a provider adapter architecture that decouples the core server logic from individual LLM service implementations. This design pattern ensures that the router and service layers interact with any supported platform—whether Google Gemini, Groq, or OpenAI-compatible endpoints—through a single, well-typed contract. By centralizing shared concerns like timeout handling, error parsing, and streaming safety in a base class, the system remains extensible without requiring changes to the routing logic when adding new providers.

Core Components of the Provider Adapter Architecture

The architecture consists of three distinct layers that separate interface definition from implementation details.

BaseProvider Abstract Class

The foundation of the provider adapter architecture resides in server/src/providers/base.ts, where the BaseProvider abstract class defines the common contract that every adapter must fulfill.

export abstract class BaseProvider {
  abstract readonly platform: Platform;
  abstract readonly name: string;
  keyless = false;                     // true for free‑tier gateways

  abstract chatCompletion(
    apiKey: string,
    messages: ChatMessage[],
    modelId: string,
    options?: CompletionOptions,
    quotaContext?: QuotaObservationContext,
  ): Promise<ChatCompletionResponse>;

  abstract streamChatCompletion(
    apiKey: string,
    messages: ChatMessage[],
    modelId: string,
    options?: CompletionOptions,
    quotaContext?: QuotaObservationContext,
  ): AsyncGenerator<ChatCompletionChunk>;

  abstract validateKey(apiKey: string, quotaContext?: QuotaObservationContext): Promise<KeyValidationResult>;

  // ── shared helpers (timeout, error parsing, stall watchdog, etc.) ──
}

All concrete adapters inherit shared utilities from this base class, including fetchWithTimeout for consistent timeout behavior, providerHttpError for back-off parsing, and readWithStallTimeout for streaming safety. This guarantees uniform error handling and resilience characteristics across every LLM integration.

Concrete Provider Adapters

Each LLM service extends BaseProvider to implement the three abstract methods using provider-specific wire formats. The GoogleProvider in server/src/providers/google.ts demonstrates this pattern for the Gemini API:

export class GoogleProvider extends BaseProvider {
  readonly platform = 'google' as const;
  readonly name = 'Google Gemini';

  async chatCompletion(...) { /* build Gemini request, call fetchWithTimeout */ }
  async *streamChatCompletion(...) { /* read SSE, apply readWithStallTimeout */ }
  async validateKey(...) { /* probe /v1/models or /v1/health */ }
}

This file contains Gemini-specific logic such as signature caching and system-prompt folding while reusing the generic helpers from BaseProvider. Other implementations like OpenAICompatProvider (server/src/providers/openai-compat.ts), CohereProvider, and CloudflareProvider follow identical structural patterns, differing only in endpoint URLs, request shaping, and provider-specific quirks.

Provider Registry

The central registry in server/src/providers/index.ts maintains a singleton Map<Platform, BaseProvider> that stores instantiated adapters. Registration occurs at module load time:

const providers = new Map<Platform, BaseProvider>();

function register(provider: BaseProvider) {
  providers.set(provider.platform, provider);
}

// Example registration
register(new GoogleProvider({ timeoutMs: 60_000 }));
register(new OpenAICompatProvider({ platform: 'groq', name: 'Groq', baseUrl: 'https://api.groq.com/openai/v1' }));

The router retrieves adapters using providers.get(platform), ensuring the service layer never contains provider-specific conditional logic.

How the Architecture Processes Requests

When handling a chat completion request, the router resolves the appropriate adapter from the registry and delegates execution:

import { providers } from '@/server/src/providers/index.js';

// Resolve the adapter
const provider = providers.get(request.platform);
if (!provider) throw new Error('Unsupported platform');

// Perform a non‑streaming completion
const response = await provider.chatCompletion(
  apiKey,
  messages,
  modelId,
  { temperature: 0.7, max_tokens: 512 },
);

The same abstraction applies to streaming responses via streamChatCompletion() and API key validation through validateKey(). The returned ChatCompletionResponse remains uniform regardless of whether the underlying provider uses OpenAI-compatible JSON or Gemini's proprietary format.

Extending the Provider Adapter Architecture

Adding support for a new LLM service requires only two steps:

1. Implement the contract: Create a class extending BaseProvider in a new file under server/src/providers/, implementing the three abstract methods with the provider's specific HTTP logic.
2. Register the instance: Import the class into server/src/providers/index.ts and call register(new MyNewProvider()) to add it to the singleton map.

This modular approach allows developers to introduce new platforms without modifying the router, validation middleware, or quota tracking systems.

Summary

• Abstract Contract: The BaseProvider class in server/src/providers/base.ts defines the mandatory interface (chatCompletion, streamChatCompletion, validateKey) and shared utilities for timeout and error handling.
• Concrete Implementations: Provider-specific adapters like GoogleProvider and OpenAICompatProvider extend the base class to handle unique wire formats while inheriting common resilience patterns.
• Central Registry: A Map<Platform, BaseProvider> in server/src/providers/index.ts manages singleton instances and enables runtime lookup by platform identifier.
• Router Isolation: The service layer retrieves adapters via providers.get(platform), ensuring complete decoupling from provider-specific implementation details.
• Extensibility: New LLM services require only a new adapter class and a single registration call, with no changes needed to the core routing logic.

Frequently Asked Questions

What is the role of BaseProvider in FreeLLMAPI?

BaseProvider serves as the abstract contract that enforces uniformity across all LLM integrations. It defines three required methods—chatCompletion, streamChatCompletion, and validateKey—while providing inherited helpers like fetchWithTimeout and readWithStallTimeout to ensure consistent timeout behavior and error handling across every provider.

How does the provider registry manage adapter instances?

The registry in server/src/providers/index.ts maintains a singleton Map<Platform, BaseProvider> that stores one instance of each concrete adapter. The register() function populates this map at module load time, mapping platform identifiers (like 'google' or 'groq') to their respective adapter instances for runtime resolution.

What methods must a concrete provider implement?

Every concrete adapter must implement three abstract methods defined in BaseProvider: chatCompletion() for non-streaming requests, streamChatCompletion() for streaming responses (returning an AsyncGenerator), and validateKey() for API key verification. These implementations handle provider-specific HTTP calls while returning standardized response types.

How does the architecture handle provider-specific errors?

The BaseProvider class includes shared error parsing utilities such as providerHttpError that concrete adapters invoke when processing HTTP responses. This centralizes back-off logic and error classification, allowing the router to handle failures uniformly regardless of whether the underlying provider returns OpenAI-style error objects or Gemini-specific status codes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}