How to Add a New Provider Adapter to FreeLLMAPI

To add a new provider adapter to FreeLLMAPI, extend either OpenAICompatProvider (for OpenAI-compatible APIs) or BaseProvider (for custom implementations), then register the instance in server/src/providers/index.ts and optionally seed free-tier models in server/src/db/migrations.ts.

FreeLLMAPI routes all LLM traffic through a unified provider-adapter layer that abstracts differences between upstream services. Adding a new provider to FreeLLMAPI requires implementing a common interface and registering it in the central registry, allowing the router in server/src/services/router.ts to immediately utilize the new service for chat completions and streaming.

Understanding the Provider Architecture

The BaseProvider Contract

Every adapter must satisfy the BaseProvider abstract class defined in server/src/providers/base.ts. This contract mandates three core methods: chatCompletion() for non-streaming requests, streamChatCompletion() for SSE streaming, and validateKey() for API key verification. The class also provides shared utilities for HTTP timeouts, SSE reading, and standardized error handling.

export abstract class BaseProvider {
  abstract readonly platform: Platform;
  abstract readonly name: string;
  abstract chatCompletion(...): Promise<ChatCompletionResponse>;
  abstract streamChatCompletion(...): AsyncGenerator<ChatCompletionChunk>;
  abstract validateKey(...): Promise<boolean>;
  // …timeout, SSE helpers, etc.
}

OpenAICompatProvider for Compatible Services

For services that already implement the OpenAI API specification, server/src/providers/openai-compat.ts offers a concrete implementation called OpenAICompatProvider. This class handles request building, response normalization, and tool-call rescue logic automatically. Providers like Groq, NVIDIA, and Mistral use this base class to avoid boilerplate code.

The Provider Registry

The central registry in server/src/providers/index.ts maintains a singleton Map<Platform, BaseProvider> that maps platform identifiers to adapter instances. The register() helper adds providers to this map, and the exported getProvider() function resolves the correct instance during request routing. Registration order determines the default fallback chain priority when the router selects providers.

Step 1 – Create the Provider Adapter

Option A: Extend OpenAICompatProvider

If the upstream service uses an OpenAI-compatible HTTP API, create a subclass of OpenAICompatProvider. The constructor must specify the platform identifier, display name, base URL, and optional custom headers.

Option B: Implement BaseProvider Directly

For non-compatible services, implement the abstract BaseProvider class directly. You must provide concrete implementations for chatCompletion(), streamChatCompletion(), and validateKey(), handling the specific authentication and response formats of the upstream API.

Step 2 – Register the Adapter in the Registry

Import your new provider class into server/src/providers/index.ts and invoke register(new YourProvider()) alongside existing registrations. The position in the file determines priority in the fallback chain, with earlier registrations receiving higher precedence during routing decisions.

Step 3 – Seed Free-Tier Models (Optional)

If the provider offers free models, add them to the database via migration scripts in server/src/db/migrations.ts. Create a function that inserts rows into the models table with the appropriate platform, model_id, is_free flag, and quota limits, then invoke it from the main migrateModels() function. The router automatically discovers these entries at runtime.

Implementation Examples

Adding an OpenAI-Compatible Provider

Create a new file under server/src/providers/ for your adapter:

// file: server/src/providers/acme.ts
import { OpenAICompatProvider } from './openai-compat.js';

export class AcmeProvider extends OpenAICompatProvider {
  constructor() {
    super({
      platform: 'acme',
      name: 'AcmeAI',
      baseUrl: 'https://api.acme.ai/v1',
      extraHeaders: {
        'X-Requested-By': 'FreeLLMAPI',
      },
    });
  }
}

Registering the Provider

Add the import and registration to the central registry:

// file: server/src/providers/index.ts
import { AcmeProvider } from './acme.js';

// Existing registrations...
register(new AcmeProvider());

Adding Free-Tier Model Rows

If AcmeAI offers free models, expose them via the migration system:

// file: server/src/db/migrations.ts
function migrateModelsV30AcmeFree(db: Database.Database) {
  db.prepare(`
    INSERT OR IGNORE INTO models (platform, model_id, family, is_free, quota_limit)
    VALUES ('acme', 'acme-phi-3-mini', 'phi-3-mini', 1, 100000)
  `).run();
}

Then call migrateModelsV30AcmeFree(db) from the top-level migrateModels() function.

Summary

• FreeLLMAPI uses a BaseProvider abstract class in server/src/providers/base.ts to standardize LLM interactions across different services.
• For OpenAI-compatible APIs, extend OpenAICompatProvider from server/src/providers/openai-compat.ts to inherit request handling, streaming, and error-rescue logic.
• Register every adapter in server/src/providers/index.ts using the register() function; registration order defines the fallback priority used by the router.
• The router in server/src/services/router.ts automatically integrates registered providers into the request lifecycle without additional configuration.
• Optional free-tier models can be seeded in server/src/db/migrations.ts to expose specific provider models to users via the database catalog.

Frequently Asked Questions

What is the difference between BaseProvider and OpenAICompatProvider?

BaseProvider is the abstract contract requiring manual implementation of chatCompletion(), streamChatCompletion(), and validateKey() for non-standard APIs. OpenAICompatProvider is a concrete subclass that handles OpenAI-compatible services automatically, requiring only configuration of the base URL and headers.

How does the router choose which provider to use?

The router in server/src/services/router.ts consults the registry map in server/src/providers/index.ts to resolve providers by platform. It evaluates the fallback chain order, API key health, and rate limits to select the appropriate instance for each request.

Can I add a provider without modifying the database migrations?

Yes. Database seeding is only required if you want to expose free-tier models to users. The adapter will function for direct API calls using the platform identifier even without migration entries, though the models won't appear in the free-tier catalog.

Where should I place custom error handling logic?

For OpenAI-compatible providers, error handling is inherited from OpenAICompatProvider. For custom implementations, override the error handling methods or intercept responses within your chatCompletion() and streamChatCompletion() implementations in your BaseProvider subclass.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}