How to Configure FreeLLMAPI Provider Adapters: Environment Variables, Custom Adapters, and Routing

FreeLLMAPI configures provider adapters through environment variables defined in .env files and a modular TypeScript class hierarchy centered on BaseProvider in server/src/providers/base.ts.

FreeLLMAPI (tashfeenahmed/freellmapi) abstracts multiple LLM backends behind a unified interface using a provider-adapter pattern. Configuring FreeLLMAPI provider adapters involves setting provider-specific API keys in environment variables for existing integrations, while adding new providers requires extending the base adapter class and registering it in the provider index. This guide walks through both configuration strategies using actual source paths and implementation details from the repository.

Architecture of FreeLLMAPI Provider Adapters

The system decouples provider-specific logic from request handling through an adapter pattern. Each supported LLM platform (OpenAI, Anthropic, Groq, Google, etc.) is implemented as a standalone TypeScript module in server/src/providers/ that inherits from the abstract BaseProvider class defined in server/src/providers/base.ts.

The router (server/src/services/router.ts) acts as the central dispatch layer. When a request arrives, the router identifies the target platform, instantiates the corresponding adapter, and delegates to either chatCompletion() or streamChatCompletion(). The adapter translates the unified internal request format into provider-specific wire protocols, manages authentication headers, and handles streaming responses. Quota enforcement and rate-limiting logic reside in server/src/services/provider-quota.ts, which adapters interact with to report usage.

Configuring Existing Adapters via Environment Variables

All built-in adapters read configuration from environment variables at startup. The repository includes an .env.example file documenting the required variables for each provider.

Standard configuration uses prefixed variable names:

• OPENAI_API_KEY, OPENAI_ENDPOINT, OPENAI_MODEL
• ANTHROPIC_API_KEY, ANTHROPIC_ENDPOINT
• GROQ_API_KEY, GROQ_ENDPOINT

Additional tuning parameters control quota behavior:

• PROVIDER_QUOTA_STRATEGY — Strategy for allocating quota across keys
• PROVIDER_RPM_LIMIT — Requests-per-minute ceiling per adapter
• PROVIDER_TPM_LIMIT — Tokens-per-minute ceiling per adapter

# .env configuration for OpenAI and Anthropic adapters

OPENAI_API_KEY=sk-your-openai-key
OPENAI_ENDPOINT=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o

ANTHROPIC_API_KEY=sk-ant-your-key
ANTHROPIC_ENDPOINT=https://api.anthropic.com/v1

How the Router Uses Provider Adapters

The routing layer (server/src/services/router.ts) selects the appropriate adapter based on the platform identifier in the incoming request. It imports adapter instances from server/src/providers/index.ts and forwards the call to the unified interface methods.

// Conceptual flow based on server/src/services/router.ts
import { getProvider } from '../providers/index.js';

async function handleRequest(request) {
  const provider = getProvider(request.platform);
  if (!provider) {
    throw new Error(`Unsupported platform ${request.platform}`);
  }
  
  // Delegates to the adapter's implementation
  return await provider.chatCompletion(request);
}

Each adapter implements two primary methods defined in BaseProvider:

• chatCompletion(req: ProviderRequest): Promise<ProviderResponse> — For standard blocking completions
• streamChatCompletion(req: ProviderRequest): Promise<ReadableStream> — For Server-Sent Events streaming

Implementing a Custom Provider Adapter

Adding a new LLM backend requires four steps:

1. Create a new file server/src/providers/<provider>.ts extending BaseProvider
2. Implement chatCompletion and streamChatCompletion methods
3. Add environment variables to .env.example
4. Register the adapter in server/src/providers/index.ts

Extending BaseProvider

Create a class that inherits from BaseProvider located at server/src/providers/base.ts. This enforces type consistency across all adapters.

// server/src/providers/myprovider.ts
import { BaseProvider } from './base.js';
import type { ProviderRequest, ProviderResponse } from '../types.js';

export class MyProvider extends BaseProvider {
  // Implementation required
}

Implementing Completion Methods

The adapter must translate the internal ProviderRequest format into the external provider's JSON schema and map responses back to the standard ProviderResponse type defined in shared/types.ts.

async chatCompletion(req: ProviderRequest): Promise<ProviderResponse> {
  // 1. Apply authentication from environment variables
  const headers = { 
    Authorization: `Bearer ${process.env.MYPROVIDER_API_KEY}` 
  };
  
  // 2. Transform request payload to provider format
  const body = { 
    prompt: req.messages.map(m => m.content).join('\n') 
  };
  
  // 3. Execute HTTP request
  const response = await fetch(process.env.MYPROVIDER_ENDPOINT, {
    method: 'POST',
    headers,
    body: JSON.stringify(body)
  });
  
  // 4. Normalize response to ProviderResponse interface
  const data = await response.json();
  return this.standardizeResponse(data);
}

Registering the Adapter

Export the new adapter in the provider index so the router can discover it:

// server/src/providers/index.ts
import { OpenAIProvider } from './openai.js';
import { AnthropicProvider } from './anthropic.js';
import { MyProvider } from './myprovider.js';

export const ADAPTERS = [
  new OpenAIProvider(),
  new AnthropicProvider(),
  new MyProvider()  // New addition
];

Managing Quotas and Rate Limits

The server/src/services/provider-quota.ts module tracks token and request consumption across adapters. Adapters report usage metrics after each API call, enabling centralized enforcement of limits defined by PROVIDER_RPM_LIMIT and PROVIDER_TPM_LIMIT.

When an adapter approaches its configured threshold, the quota service can trigger back-off strategies or reject requests before they reach the upstream provider. This prevents API key suspensions and controls costs across multiple configured backends.

Summary

• FreeLLMAPI provider adapters are TypeScript classes extending BaseProvider in server/src/providers/.
• Configuration occurs through environment variables prefixed with the provider name (e.g., OPENAI_API_KEY), documented in .env.example.
• The router (server/src/services/router.ts) dispatches requests to adapters using the chatCompletion() and streamChatCompletion() methods.
• Custom adapters require implementing the base interface, then registering the instance in server/src/providers/index.ts.
• Quotas and rate limits are enforced by server/src/services/provider-quota.ts using variables like PROVIDER_RPM_LIMIT and PROVIDER_TPM_LIMIT.

Frequently Asked Questions

What environment variables are required to configure a provider adapter?

Each adapter requires an API key variable (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY) and typically an endpoint URL (e.g., OPENAI_ENDPOINT). Optional tuning variables include PROVIDER_RPM_LIMIT for rate limiting and PROVIDER_QUOTA_STRATEGY for quota allocation. These are defined in the repository's .env.example file.

How do I add a new LLM provider that isn't officially supported?

Create a new file in server/src/providers/ that extends BaseProvider from base.ts, implement the chatCompletion and streamChatCompletion methods to handle HTTP translation and authentication, add the required environment variables to .env.example, and export the new class in server/src/providers/index.ts so the router can load it.

Where does FreeLLMAPI handle rate limiting for provider adapters?

Rate limiting and quota management logic reside in server/src/services/provider-quota.ts. This service tracks requests-per-minute (RPM) and tokens-per-minute (TPM) across all configured adapters, enforcing limits defined by environment variables before requests reach the upstream LLM providers.

Can I configure multiple API keys for the same provider adapter?

Yes. While the basic configuration uses a single environment variable per provider, the adapter architecture supports key rotation and load balancing by modifying the adapter implementation to read multiple keys (e.g., OPENAI_API_KEY_1, OPENAI_API_KEY_2) and selecting one based on the quota strategy defined in PROVIDER_QUOTA_STRATEGY.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}