How to Add a New LLM Provider to FreeLLMAPI: Complete Implementation Guide

To add a new LLM provider to FreeLLMAPI, create a provider class extending BaseProvider (typically copying OpenAICompatProvider), register it in server/src/providers/index.ts, seed free-tier models via database migrations in server/src/db/migrations/, and add a unit test in server/src/__tests__/providers/.

FreeLLMAPI is an open-source gateway that aggregates free-tier LLM APIs through a unified interface. Adding a new LLM provider to FreeLLMAPI involves implementing the provider interface, registering it with the central provider registry, and populating the model catalog so the router and UI can discover available endpoints.

Understanding the Provider Architecture

FreeLLMAPI uses a provider registry pattern centered in server/src/providers/index.ts. The architecture relies on two core abstractions:

• BaseProvider (server/src/providers/base.ts): The abstract base class defining the provider contract for chat completions, embeddings, and health checks.
• OpenAICompatProvider (server/src/providers/openai-compat.ts): A concrete implementation handling OpenAI-compatible endpoints, including streaming, tool-call rescue, quota tracking, and timeout handling.

The provider registry maintains a Map<Platform, BaseProvider> that stores singleton instances. The router (server/src/services/router.ts) queries this registry via getProvider(platform) to route requests, perform fail-over, and enforce per-key quotas.

Step-by-Step Guide to Adding a New Provider

Create the Provider Implementation

Most new providers should extend OpenAICompatProvider rather than implementing BaseProvider from scratch. This template already implements complex logic for authentication, streaming, and error handling.

Copy the template file:

cp server/src/providers/openai-compat.ts server/src/providers/mycloud.ts

Modify the class definition:

// server/src/providers/mycloud.ts
import { BaseProvider } from './base.js';
import type { Platform } from '@freellmapi/shared/types.js';

export class MyCloudProvider extends BaseProvider {
  readonly platform: Platform = 'mycloud';
  readonly name = 'MyCloud';
  private readonly baseUrl = 'https://api.mycloud.ai/v1';

  constructor() {
    super();
    // Configure provider-specific options here
  }
}

Register the Provider in the Central Registry

In server/src/providers/index.ts, import your new class and add it to the registry:

// server/src/providers/index.ts
import { MyCloudProvider } from './mycloud.js';

// Add to existing registrations
register(new MyCloudProvider());

The register() helper adds the instance to the internal Map, making the platform string discoverable by the router and health-check logic. Without this step, requests targeting platform: 'mycloud' will be rejected with a "no provider registered" error.

Seed the Model Catalog

FreeLLMAPI stores the static list of free-tier models in database migration scripts under server/src/db/migrations/. Create a migration function that inserts rows for each available model:

// server/src/db/migrations/20260101_000000_legacy_baseline.ts
export function migrateModelsVXXMyCloud(db: Database.Database) {
  const stmt = db.prepare(`
    INSERT INTO models (platform, model_id, display_name, provider_id, quota_limit)
    VALUES ('mycloud', 'mycloud/gpt-4-free', 'MyCloud GPT-4 Free', 'mycloud', 100000)
  `);
  stmt.run();
}

Call this function from the main migration runner (around line 10 of the baseline migration file). The UI's Models page displays only rows existing in the models table, making this step essential for visibility.

Add Required Unit Tests

FreeLLMAPI's CI pipeline enforces 100% test coverage for providers. Create a test file verifying registration and model lookup:

// server/src/__tests__/providers/mycloud.test.ts
import { hasProvider, getProvider } from '../../src/providers/index.js';
import { expect, test } from 'vitest';

test('mycloud provider registration', () => {
  expect(hasProvider('mycloud')).toBe(true);
  const prov = getProvider('mycloud');
  expect(prov?.name).toBe('MyCloud');
});

Run the test suite with npm test to ensure the provider integrates correctly with the registry and router.

Integration Points and Key Files

Understanding where your provider fits in the system ensures proper integration:

• Router (server/src/services/router.ts): Selects providers based on platform keys, performs fail-over between endpoints, and tracks quota consumption.
• Health Checks (server/src/services/health.ts): Periodically validates API keys and marks them as healthy or invalid based on provider responses.
• Database Migrations (server/src/db/migrations/): Single source of truth for which models are considered free-tier for each provider.

Summary

• Extend OpenAICompatProvider in server/src/providers/ to inherit streaming, tool-call handling, and quota tracking logic.
• Register the provider using register(new YourProvider()) in server/src/providers/index.ts to make it discoverable.
• Seed model data via migration functions in server/src/db/migrations/ to populate the UI and router catalog.
• Add unit tests in server/src/__tests__/providers/ to maintain CI compliance and verify registration.
• Update documentation in README.md to list the new supported provider.

Frequently Asked Questions

Can I implement a custom provider without using OpenAICompatProvider?

Yes, you can extend BaseProvider directly from server/src/providers/base.ts if your LLM API uses a non-OpenAI protocol. However, you must implement all abstract methods including chatCompletion, streamChatCompletion, and error handling logic that OpenAICompatProvider provides by default.

Where does FreeLLMAPI store the list of available models?

The model catalog lives in the SQLite database, seeded via migration scripts in server/src/db/migrations/. Each row maps a model_id to its platform, display_name, and quota_limit. The UI queries this table to display available free-tier models.

Why is my new provider not appearing in the UI?

If the provider is registered but invisible in the UI, check that you have run the database migration to insert model rows. The UI displays only models present in the models table. Additionally, verify that hasProvider('your-platform') returns true in your test file.

How does the router handle authentication for new providers?

Authentication is handled within the provider implementation. When extending OpenAICompatProvider, the class manages API keys via the Authorization header and supports custom headers through the extraHeaders constructor option. The router (server/src/services/router.ts) selects which API key to use based on health status and quota availability, but delegates the actual request signing to the provider class.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}