How to Contribute a New LLM Provider to FreeLLMAPI: A Complete Developer's Guide

To contribute a new LLM provider to FreeLLMAPI, extend the abstract BaseProvider class in server/src/providers/base.ts, implement the required service methods (generate, validateKey, and optionally embedding and healthCheck), register your provider in server/src/providers/index.ts, and configure quota header mapping in server/src/services/provider-quota.ts if your provider exposes rate-limit metadata.

FreeLLMAPI is an open-source unified gateway that normalizes multiple large language model services behind a single API interface. Contributing a new provider means implementing a standardized adapter that the central router can consume for request dispatching and failover handling. This guide details the exact file locations, class structures, and registry patterns used in the tashfeenahmed/freellmapi repository.

Understanding the Provider Architecture

FreeLLMAPI abstracts every external LLM service through a common interface defined in server/src/providers/base.ts. The BaseProvider class specifies the contract that all concrete implementations must follow, including methods for text generation, embedding, key validation, and health checks.

The Provider Registry in server/src/providers/index.ts maintains a map of platform identifiers (e.g., "openai-compat", "anthropic") to instantiated provider objects. Exports like getProvider, hasProvider, and resolveProvider allow the Router (server/src/services/router.ts) to look up and dispatch requests to the correct implementation.

Supporting services include Provider Quota (server/src/services/provider-quota.ts), which parses rate-limit headers from provider responses, and Model Groups (server/src/services/model-groups.ts), which collapses identical logical models across multiple providers for the UI and failover logic.

Step-by-Step Implementation Guide

1. Create the Provider Implementation

Create a new TypeScript file under server/src/providers/ (e.g., myprovider.ts). Import BaseProvider from ./base.js and implement all required abstract methods:

import { BaseProvider } from './base.js';
import type { GenerateArgs, EmbeddingArgs } from '../types.js';

export class MyProvider extends BaseProvider {
  async generate(args: GenerateArgs) {
    // Translate standard args into provider-specific HTTP request
    const resp = await fetch('https://api.myprovider.com/v1/completions', {
      method: 'POST',
      headers: { 'Authorization': `Bearer ${this.apiKey}` },
      body: JSON.stringify({ prompt: args.prompt, temperature: args.temperature }),
    });
    const json = await resp.json();
    return { text: json.choices[0].text };
  }

  async embedding(args: EmbeddingArgs) {
    // Optional: implement for embedding model support
    const resp = await fetch('https://api.myprovider.com/v1/embeddings', {
      method: 'POST',
      headers: { 'Authorization': `Bearer ${this.apiKey}` },
      body: JSON.stringify({ input: args.input }),
    });
    const json = await resp.json();
    return { embedding: json.data[0].embedding };
  }

  async validateKey(key: string) {
    // Verify API key validity (used by UI "Check key" feature)
    const resp = await fetch('https://api.myprovider.com/v1/health', {
      headers: { 'Authorization': `Bearer ${key}` },
    });
    return resp.ok;
  }

  async healthCheck() {
    // Return health status for monitoring
    return { status: 'healthy', timestamp: new Date().toISOString() };
  }
}

Reference the OpenAI compatibility implementation in server/src/providers/openai-compat.ts for a complete production example.

2. Register the Provider

Edit server/src/providers/index.ts to import your class and add it to the providers record:

import { MyProvider } from './myprovider.js';
import { BaseProvider } from './base.js';

const providers: Record<string, BaseProvider> = {
  // existing entries...
  'myprovider': new MyProvider(),
};

export { providers, getProvider, hasProvider, resolveProvider };

The string key (myprovider) becomes the platform identifier that users pass in API requests via the platform parameter.

3. Configure Quota Tracking

If your provider returns rate-limit headers, update server/src/services/provider-quota.ts. Add a mapping entry to the header configuration (often found in a KNOWN_HEADERS array or similar structure):

{
  metric: 'requests',
  limit: 'x-myprovider-limit-requests',
  remaining: 'x-myprovider-remaining-requests',
  reset: 'x-myprovider-reset-requests',
  strategy: 'provider_reported'
}

This enables the generic rate-limiting system to automatically track provider usage and store observations in the provider_quota tables.

4. Define Model Metadata

Create seed entries in the model_pricing table using migration scripts located under server/src/db/. The model-groups service automatically groups these rows with identical logical models from other providers, enabling failover and UI deduplication.

Testing Your Implementation

Validate your provider against the existing test suite to ensure compatibility with the routing and rate-limiting systems:

npm test

The test files server/src/__tests__/services/router.test.ts and server/src/__tests__/services/ratelimit.test.ts exercise providers through the central router, verifying that your implementation satisfies the BaseProvider contract and correctly handles quota headers.

Using the New Provider

Once registered and deployed, users can invoke your provider via the API:

curl -X POST https://api.freellmapi.com/v1/generate \
  -H "Authorization: Bearer YOUR_FREE_LLMAPI_KEY" \
  -d '{
    "platform": "myprovider",
    "model": "gpt-mini",
    "prompt": "Explain quantum entanglement in one sentence."
  }'

Summary

• Extend BaseProvider in server/src/providers/base.ts to create a standardized adapter for your LLM service, implementing generate, validateKey, and optional methods.
• Register in server/src/providers/index.ts using a unique platform identifier to make the provider available to the router's getProvider and resolveProvider functions.
• Configure quota parsing in server/src/services/provider-quota.ts if your provider exposes rate-limit headers, mapping them to the standard metric fields.
• Add model metadata to the database migrations under server/src/db/ so the model-groups service can handle logical grouping across providers.
• Run tests in server/src/__tests__/services/router.test.ts to verify integration with the routing, rate-limiting, and failover systems.

Frequently Asked Questions

What methods are mandatory when extending BaseProvider?

You must implement generate for text generation and validateKey for API key authentication. Optional methods include embedding for vector embeddings and healthCheck for service monitoring. The abstract class definition in server/src/providers/base.ts defines the exact signatures required, and server/src/providers/openai-compat.ts provides a reference implementation.

How does FreeLLMAPI route requests to my new provider?

The router in server/src/services/router.ts calls resolveProvider from server/src/providers/index.ts, passing the platform parameter from the incoming request. Ensure your provider is registered in the providers map with the exact key users specify (e.g., "myprovider"), or the router will return a "provider not found" error.

Where do I document environment variables for my provider?

Add documentation to CONTRIBUTING.md and the main README, specifying required environment variables such as MYPROVIDER_API_KEY. Access these variables in your provider class via this.apiKey (set during instantiation) or process.env, and use validateKey to verify credentials against your provider's authentication endpoint.

How do I handle different rate-limiting schemes?

Map your provider's specific HTTP headers in server/src/services/provider-quota.ts. Create an entry defining the metric (requests or tokens), the header names for limit, remaining, and reset, and set the strategy to 'provider_reported'. This allows the generic rate-limiting service in server/src/services/ratelimit.ts to track usage automatically without provider-specific logic in the router.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}