# How to Set Up FreeLLMAPI with Custom Providers: A Complete Guide

> Learn to set up FreeLLMAPI with custom providers easily. Connect to any OpenAI-compatible endpoint and encrypt credentials securely for load balancing and health monitoring.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-27

---

**You can configure FreeLLMAPI to route requests to any OpenAI-compatible endpoint by registering a custom provider through the dashboard UI or the `resolveCustomEndpointKey` function, which encrypts your credentials in SQLite and exposes them to the router for automatic load balancing and health monitoring.**

FreeLLMAPI is an open-source AI model router that aggregates free-tier providers into a unified API. When you need to integrate private endpoints or self-hosted models, understanding how to set up FreeLLMAPI with custom providers allows you to extend the platform beyond its pre-bundled services while maintaining the same quota tracking and failover logic.

## Architecture Overview

The system treats custom endpoints as a distinct **platform** named `custom` within the SQLite catalog. Each entry consists of a base URL, optional secret key, and metadata stored in the encrypted `api_keys` table.

In [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), the router queries the catalog via `getProvider` → `BaseProvider` → [`openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/openai-compat.ts) to decrypt secrets and forward requests. This architecture ensures that client applications only need the unified bearer token (`freellmapi-…`), while the server handles upstream authentication transparently.

## Registering a Custom Provider

You can register custom endpoints through the web dashboard, CLI, or programmatically via the TypeScript API.

### Dashboard Method

Navigate to **Settings → Custom Endpoints** in the admin UI and click **Add Endpoint**. Enter the base URL (e.g., `https://my-relay.example.com/v1`) and optional API key, assign a label, then save. The UI calls `resolveCustomEndpointKey` in [`server/src/services/custom-endpoint.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/custom-endpoint.ts) to persist the configuration with AES-256-GCM encryption.

### CLI Method

Use the `freellmapi doctor` command with the `--custom-endpoint` flag:

```bash
freellmapi doctor --custom-endpoint "https://my-relay.local/v1" --secret "sk-mysecret"

```

This command invokes the same registration logic as the dashboard, creating or updating the credential row in the database.

## Programmatic Configuration

For automated deployments or configuration-as-code, import `resolveCustomEndpointKey` directly from the server source:

```typescript
import { resolveCustomEndpointKey } from
  'https://github.com/tashfeenahmed/freellmapi/raw/main/server/src/services/custom-endpoint.ts';
import { Database } from 'better-sqlite3';

// 1️⃣ Initialise the DB (the same DB used by the running server)
const db = new Database('freellmapi.db');

// 2️⃣ Register (or update) a custom endpoint
const result = resolveCustomEndpointKey(
  db,
  'https://my-relay.example.com/v1', // base URL
  'sk‑my‑custom‑key',                // secret (or undefined if none)
  'My Private Relay',                // optional label
  undefined                            // pinnedKeyId – not needed here
);

console.log('Endpoint registration result:', result);
// { keyId: 42, storedKey: 'sk‑my‑custom‑key', created: true }

```

After registration, add a model in the **Models** panel by selecting **Platform → custom** and choosing your endpoint from the dropdown. The model inherits the endpoint’s quota and health status.

## Routing and Rate Limiting

When a request arrives for a custom model, the router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) decrypts the stored secret and forwards the request via [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts). The [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) service applies the same RPM (requests per minute) and TPM (tokens per minute) caps to custom providers as it does to built-in ones.

If the custom endpoint returns a 429 error, the system places the key on cooldown according to the logic in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) and automatically retries the next available model in the fallback chain.

## Security and Health Monitoring

All secrets are encrypted at rest using AES-256-GCM before storage in the `api_keys` table. Decryption occurs only in memory immediately before a request is forwarded to the upstream provider.

The [`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts) service polls custom endpoints continuously, updating status flags (`healthy`, `rate_limited`, `invalid`) in the catalog. Quota usage appears in the **Analytics** tab, allowing you to monitor custom provider consumption alongside built-in models.

## Summary

- **FreeLLMAPI** stores custom provider configurations as encrypted entries in the SQLite `api_keys` table under the `custom` platform namespace.
- Use **`resolveCustomEndpointKey`** in [`server/src/services/custom-endpoint.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/custom-endpoint.ts) to register endpoints programmatically or via the dashboard/CLI.
- The **router** ([`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)) and **rate limiter** ([`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts)) treat custom providers identically to built-in ones, ensuring consistent failover and quota enforcement.
- Credentials are **encrypted at rest** with AES-256-GCM and only decrypted in memory during request forwarding.
- The **health probe** ([`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts)) monitors custom endpoints and updates status flags without exposing upstream keys to clients.

## Frequently Asked Questions

### Do I need to modify server configuration files to add custom providers?

No. FreeLLMAPI stores all custom endpoint configurations dynamically in the SQLite database. You interact with the system exclusively through the admin dashboard, CLI commands, or the programmatic API, eliminating the need for YAML or JSON configuration files.

### Can I register multiple API keys for the same custom endpoint URL?

Yes. The system supports credential pooling where each key gets its own row in the database. The router can rotate between multiple keys for the same base URL, distributing load and providing redundancy if one key hits rate limits.

### What happens when a custom provider returns a 429 status code?

The router applies standard cooldown logic from [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts), marking the specific key as rate-limited and temporarily removing it from the rotation. Subsequent requests automatically fall back to the next available model in the chain until the cooldown period expires.

### Is the upstream API key visible to client applications using the FreeLLMAPI endpoint?

No. Client applications authenticate using only the unified `freellmapi-…` bearer token. The upstream secret is decrypted server-side in [`openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/openai-compat.ts) and never transmitted to or visible by end users.