How to Add a New LLM Provider to FreeLLMAPI: A Step-by-Step Implementation Guide

To add a new LLM provider to FreeLLMAPI, extend the BaseProvider class (or reuse OpenAICompatProvider), register it in server/src/providers/index.ts, seed free-tier model entries in the database migrations, and add a unit test in server/src/__tests__/providers/.

FreeLLMAPI is an open-source LLM gateway maintained by tashfeenahmed/freellmapi that aggregates free-tier models from multiple providers through a unified API. Its architecture relies on a central provider registry in server/src/providers/index.ts that maps platform identifiers to provider implementations, handling authentication, request routing, quota tracking, and health monitoring automatically. Adding a new provider requires implementing the provider interface, registering the instance, updating the database catalog, and verifying functionality with tests.

Understanding the Provider Architecture

FreeLLMAPI’s provider system is built around an abstract BaseProvider class defined in server/src/providers/base.ts. This interface standardizes how the platform handles chat completions, embeddings, and streaming across different LLM backends. Most new providers can leverage the OpenAICompatProvider implementation in server/src/providers/openai-compat.ts, which already encapsulates complex logic for timeout handling, tool-call rescue, quota observation, and error formatting. The provider registry in server/src/providers/index.ts maintains a singleton Map<Platform, BaseProvider> that the router and health-check systems query to resolve platform strings to concrete implementations.

Step-by-Step Implementation Guide

1. Create the Provider Implementation

Start by copying the OpenAI-compatible template to create your new provider file:

cp server/src/providers/openai-compat.ts server/src/providers/mycloud.ts

Modify the file to extend BaseProvider and configure your platform-specific settings:

// server/src/providers/mycloud.ts
import { BaseProvider } from './base.js';
import type { Platform } from '@freellmapi/shared/types.js';

export class MyCloudProvider extends BaseProvider {
  readonly platform: Platform = 'mycloud';
  readonly name = 'MyCloud';
  private readonly baseUrl = 'https://api.mycloud.ai/v1';

  constructor() {
    super();
    // Configure provider-specific options like keyless access or extra headers
  }
}

2. Register the Provider in the Registry

Import your new provider class and register it in the central provider index:

// server/src/providers/index.ts
import { MyCloudProvider } from './mycloud.js';
// ... existing imports

register(new MyCloudProvider());

The register() function adds your provider instance to the internal Map, making it discoverable via getProvider('mycloud') and hasProvider('mycloud'). The router in server/src/services/router.ts uses this registry to select providers based on incoming platform keys and perform failover logic.

3. Seed the Model Catalog

FreeLLMAPI stores free-tier model metadata in the database via migration scripts. Create a new migration function in server/src/db/migrations/20260101_000000_legacy_baseline.ts to insert rows for each model you want to expose:

// server/src/db/migrations/20260101_000000_legacy_baseline.ts
export function migrateModelsVXXMyCloud(db: Database.Database) {
  const stmt = db.prepare(`
    INSERT INTO models (platform, model_id, display_name, provider_id, quota_limit)
    VALUES ('mycloud', 'mycloud/gpt-4-free', 'MyCloud GPT-4 Free', 'mycloud', 100000)
  `);
  stmt.run();
}

Call this function from the main migration runner to ensure the models appear in the UI and API responses.

4. Add Unit Tests

Create a test file to verify registration and model listing functionality:

// server/src/__tests__/providers/mycloud.test.ts
import { hasProvider, getProvider } from '../../src/providers/index.js';
import { expect, test } from 'vitest';

test('mycloud provider registration', () => {
  expect(hasProvider('mycloud')).toBe(true);
  const prov = getProvider('mycloud');
  expect(prov?.name).toBe('MyCloud');
});

FreeLLMAPI requires unit tests for every provider to maintain CI coverage standards and validate that the registration chain works correctly.

Why Each Step Matters

• Template Reuse: OpenAICompatProvider implements streaming, tool-call rescue, and quota tracking logic that would otherwise require hundreds of lines of custom code.
• Registry Entry: Without registration in server/src/providers/index.ts, the router returns "no provider registered" errors for your platform key.
• Database Seeding: The UI's Models page only displays rows existing in the models table, making migrations the source of truth for available free-tier models.
• Test Coverage: The CI pipeline enforces 100% test coverage; missing tests will fail the build.

Summary

• FreeLLMAPI uses a provider registry pattern centered in server/src/providers/index.ts to manage platform integrations.
• Copy OpenAICompatProvider from server/src/providers/openai-compat.ts as a template for OpenAI-compatible endpoints, or extend BaseProvider directly for custom implementations.
• Register your provider using the register() helper to make it available to the router and health-check systems.
• Seed model data in server/src/db/migrations/ to populate the free-tier catalog exposed in the UI.
• Write unit tests in server/src/__tests__/providers/ to verify registration and model lookup, ensuring CI compliance.

Frequently Asked Questions

Do I need to modify the router code to add a new provider?

No. The router in server/src/services/router.ts dynamically looks up providers via getProvider(platform) from the registry. As long as you register your provider in server/src/providers/index.ts, the router will automatically include it in request handling and failover logic without additional changes.

Should I extend BaseProvider or use OpenAICompatProvider?

Extend OpenAICompatProvider if your LLM platform follows the OpenAI API specification for chat completions and embeddings, as it provides built-in handling for streaming, tool calls, and error formatting. Extend BaseProvider directly only if you need completely custom request handling or use a non-standard API format.

How do I limit which models appear for the new provider?

The available models are controlled entirely by the database migration entries in server/src/db/migrations/. Only insert rows for the specific free-tier models you want to expose; the UI and API will automatically filter to show only these configured models for your platform identifier.

Why does the health check system need provider registration?

The health monitoring system in server/src/services/health.ts iterates over registered providers to validate API keys and mark them as healthy or invalid. Registration ensures your provider participates in this automated health validation cycle, preventing requests from being routed to unavailable endpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}