How to Set Up FreeLLMAPI with Custom Providers: A Complete Guide

You can configure FreeLLMAPI to route requests to any OpenAI-compatible endpoint by registering a custom provider through the dashboard UI or the resolveCustomEndpointKey function, which encrypts your credentials in SQLite and exposes them to the router for automatic load balancing and health monitoring.

FreeLLMAPI is an open-source AI model router that aggregates free-tier providers into a unified API. When you need to integrate private endpoints or self-hosted models, understanding how to set up FreeLLMAPI with custom providers allows you to extend the platform beyond its pre-bundled services while maintaining the same quota tracking and failover logic.

Architecture Overview

The system treats custom endpoints as a distinct platform named custom within the SQLite catalog. Each entry consists of a base URL, optional secret key, and metadata stored in the encrypted api_keys table.

In server/src/services/router.ts, the router queries the catalog via getProvider → BaseProvider → openai-compat.ts to decrypt secrets and forward requests. This architecture ensures that client applications only need the unified bearer token (freellmapi-…), while the server handles upstream authentication transparently.

Registering a Custom Provider

You can register custom endpoints through the web dashboard, CLI, or programmatically via the TypeScript API.

Dashboard Method

Navigate to Settings → Custom Endpoints in the admin UI and click Add Endpoint. Enter the base URL (e.g., https://my-relay.example.com/v1) and optional API key, assign a label, then save. The UI calls resolveCustomEndpointKey in server/src/services/custom-endpoint.ts to persist the configuration with AES-256-GCM encryption.

CLI Method

Use the freellmapi doctor command with the --custom-endpoint flag:

freellmapi doctor --custom-endpoint "https://my-relay.local/v1" --secret "sk-mysecret"

This command invokes the same registration logic as the dashboard, creating or updating the credential row in the database.

Programmatic Configuration

For automated deployments or configuration-as-code, import resolveCustomEndpointKey directly from the server source:

import { resolveCustomEndpointKey } from
  'https://github.com/tashfeenahmed/freellmapi/raw/main/server/src/services/custom-endpoint.ts';
import { Database } from 'better-sqlite3';

// 1️⃣ Initialise the DB (the same DB used by the running server)
const db = new Database('freellmapi.db');

// 2️⃣ Register (or update) a custom endpoint
const result = resolveCustomEndpointKey(
  db,
  'https://my-relay.example.com/v1', // base URL
  'sk‑my‑custom‑key',                // secret (or undefined if none)
  'My Private Relay',                // optional label
  undefined                            // pinnedKeyId – not needed here
);

console.log('Endpoint registration result:', result);
// { keyId: 42, storedKey: 'sk‑my‑custom‑key', created: true }

After registration, add a model in the Models panel by selecting Platform → custom and choosing your endpoint from the dropdown. The model inherits the endpoint’s quota and health status.

Routing and Rate Limiting

When a request arrives for a custom model, the router in server/src/services/router.ts decrypts the stored secret and forwards the request via server/src/providers/openai-compat.ts. The ratelimit.ts service applies the same RPM (requests per minute) and TPM (tokens per minute) caps to custom providers as it does to built-in ones.

If the custom endpoint returns a 429 error, the system places the key on cooldown according to the logic in ratelimit.ts and automatically retries the next available model in the fallback chain.

Security and Health Monitoring

All secrets are encrypted at rest using AES-256-GCM before storage in the api_keys table. Decryption occurs only in memory immediately before a request is forwarded to the upstream provider.

The health.ts service polls custom endpoints continuously, updating status flags (healthy, rate_limited, invalid) in the catalog. Quota usage appears in the Analytics tab, allowing you to monitor custom provider consumption alongside built-in models.

Summary

• FreeLLMAPI stores custom provider configurations as encrypted entries in the SQLite api_keys table under the custom platform namespace.
• Use resolveCustomEndpointKey in server/src/services/custom-endpoint.ts to register endpoints programmatically or via the dashboard/CLI.
• The router (router.ts) and rate limiter (ratelimit.ts) treat custom providers identically to built-in ones, ensuring consistent failover and quota enforcement.
• Credentials are encrypted at rest with AES-256-GCM and only decrypted in memory during request forwarding.
• The health probe (health.ts) monitors custom endpoints and updates status flags without exposing upstream keys to clients.

Frequently Asked Questions

Do I need to modify server configuration files to add custom providers?

No. FreeLLMAPI stores all custom endpoint configurations dynamically in the SQLite database. You interact with the system exclusively through the admin dashboard, CLI commands, or the programmatic API, eliminating the need for YAML or JSON configuration files.

Can I register multiple API keys for the same custom endpoint URL?

Yes. The system supports credential pooling where each key gets its own row in the database. The router can rotate between multiple keys for the same base URL, distributing load and providing redundancy if one key hits rate limits.

What happens when a custom provider returns a 429 status code?

The router applies standard cooldown logic from ratelimit.ts, marking the specific key as rate-limited and temporarily removing it from the rotation. Subsequent requests automatically fall back to the next available model in the chain until the cooldown period expires.

Is the upstream API key visible to client applications using the FreeLLMAPI endpoint?

No. Client applications authenticate using only the unified freellmapi-… bearer token. The upstream secret is decrypted server-side in openai-compat.ts and never transmitted to or visible by end users.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}