# How to Set Up Custom OpenAI-Compatible Endpoints with FreeLLMAPI > Easily set up custom OpenAI-compatible endpoints using FreeLLMAPI. Register your models once and interact seamlessly with the OpenAI API for your applications. - Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi) - Tags: how-to-guide - Published: 2026-06-28 --- **Send a POST request to `/keys/custom` with your `baseUrl` and model list, then use the registered model names in standard OpenAI API calls—FreeLLMAPI handles the rest automatically.** FreeLLMAPI is an open-source LLM gateway that unifies multiple providers behind a single OpenAI-compatible API. While it ships with built-in support for Groq, Mistral, and OpenRouter, its **custom provider system** lets you integrate any OpenAI-compatible endpoint—whether that's Ollama running locally, a self-hosted vLLM instance, or a third-party API service. This guide walks through the complete setup process based on the actual source implementation in `tashfeenahmed/freellmapi`. --- ## Understanding the Custom Provider Architecture FreeLLMAPI treats custom endpoints as **dynamic providers** that are created on-the-fly rather than initialized at startup. This differs from built-in providers like Groq or Mistral, which are instantiated once in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts). The architecture follows this flow: 1. **Registration**: Your `POST /keys/custom` request stores the endpoint configuration in the database 2. **Model binding**: Each model you specify gets inserted into the `models` table with a foreign key to your endpoint 3. **Runtime resolution**: When a chat request arrives, `resolveProvider()` constructs an `OpenAICompatProvider` instance using your stored `baseUrl` 4. **Request forwarding**: The provider forwards calls to `baseUrl + '/v1/chat/completions'` with your stored API key This design means you don't need to restart the server or modify configuration files—endpoints are immediately available after registration. --- ## Registering a Custom Endpoint ### Required Parameters The `POST /keys/custom` endpoint in [`server/src/routes/keys.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/keys.ts) expects: | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `baseUrl` | string | Yes | The OpenAI-compatible base URL (e.g., `http://localhost:11434/v1`) | | `models` | array | Yes | List of model objects or strings to register | | `apiKey` | string | No | Authentication key, or omit/`"no-key"` for keyless services | | `label` | string | No | Human-readable name for this endpoint | ### Example: Registering Ollama ```bash curl -X POST https://your-llmapi.example.com/keys/custom \ -H "Content-Type: application/json" \ -d '{ "baseUrl": "http://localhost:11434/v1", "apiKey": "no-key", "label": "Local Ollama", "models": [ {"model": "llama3:8b", "displayName": "Llama 3 8B"}, {"model": "phi2", "displayName": "Phi-2"} ] }' ``` **Response structure:** ```json { "success": true, "keyId": 23, "platform": "custom", "baseUrl": "http://localhost:11434/v1", "model": "llama3:8b", "displayName": "Llama 3 8B", "models": [ {"modelDbId": 101, "model": "llama3:8b", "displayName": "Llama 3 8B"}, {"modelDbId": 102, "model": "phi2", "displayName": "Phi-2"} ], "maskedKey": "no-key" } ``` The `keysRouter.post('/custom', ...)` handler in [`server/src/routes/keys.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/keys.ts) performs three operations: it creates or updates a row in `api_keys` with `platform = 'custom'`, normalizes the model list to handle both string and object formats, and inserts each model into the `models` table with the appropriate foreign key reference. --- ## How Provider Resolution Works at Runtime When a client sends a chat completion request using a model from your custom endpoint, FreeLLMAPI needs to determine where to route it. This happens through the `resolveProvider()` function in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts): ```typescript export function resolveProvider(platform: Platform, baseUrl?: string | null): BaseProvider | undefined { if (platform === 'custom') { const trimmed = baseUrl?.trim(); if (!trimmed) return undefined; return new OpenAICompatProvider({ platform: 'custom', name: 'Custom (OpenAI-compatible)', baseUrl: trimmed, timeoutMs: CUSTOM_PROVIDER_TIMEOUT_MS, // 30 seconds }); } return providers.get(platform); } ``` **Key details:** - Built-in providers are cached in a `Map` - Custom providers are instantiated fresh for each request using the stored `baseUrl` - Custom providers receive an extended **30-second timeout** (`CUSTOM_PROVIDER_TIMEOUT_MS`) compared to built-in providers, accommodating slower self-hosted inference The `getProviderForModel()` function handles the database lookup: it queries the `models` table by model identifier, retrieves the associated `api_keys` row including the `base_url` field, and calls `resolveProvider()` with the appropriate platform and URL. --- ## The OpenAICompatProvider Implementation The `OpenAICompatProvider` class in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) provides the actual transport layer for custom endpoints. It extends `BaseProvider` and implements four key methods: ### Authentication Handling The `authHeader()` method supports both authenticated and keyless services: ```typescript private authHeader(apiKey: string): Record { if (apiKey === 'no-key') { return {}; } return { 'Authorization': `Bearer ${apiKey}` }; } ``` When you register with `"no-key"` or omit the `apiKey` field, the provider sends requests without an `Authorization` header—essential for local services like Ollama that don't require authentication. ### Request Routing The provider normalizes the base URL by stripping trailing `/v1/` paths, then constructs full URLs: - Chat completions: `${baseUrl}/v1/chat/completions` - Model listing: `${baseUrl}/v1/models` (used for key validation) ### Timeout Configuration The constructor accepts a `timeoutMs` parameter (defaulting to 30 seconds) and uses `AbortController` to enforce it: ```typescript private async fetchWithTimeout(url: string, options: any): Promise { const controller = new AbortController(); const timeout = setTimeout(() => controller.abort(), this.timeoutMs); // ... fetch with signal } ``` --- ## Using Your Custom Endpoint Once registered, use your custom models exactly like any OpenAI-compatible model: ```bash curl -X POST https://your-llmapi.example.com/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "llama3:8b", "messages": [{"role": "user", "content": "Hello from FreeLLMAPI"}], "temperature": 0.7 }' ``` The server executes this resolution chain: 1. Looks up `"llama3:8b"` in the `models` table, finding `platform = "custom"` and `key_id = 23` 2. Retrieves the `api_keys` row for ID 23, getting `base_url = "http://localhost:11434/v1"` and `key = "no-key"` 3. Calls `resolveProvider(Platform.CUSTOM, "http://localhost:11434/v1")` 4. The `OpenAICompatProvider` forwards the request to `http://localhost:11434/v1/chat/completions` --- ## Managing Custom Endpoints ### List All Custom Endpoints ```bash curl https://your-llmapi.example.com/keys/custom ``` Returns all custom configurations with masked keys and their associated models. ### Delete a Custom Endpoint ```bash curl -X DELETE https://your-llmapi.example.com/keys/custom/23 ``` This cascades to delete associated models from the `models` table due to the foreign key constraint defined in [`server/src/db/migrations.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/migrations.ts). --- ## Database Schema for Custom Providers The migration in [`server/src/db/migrations.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/migrations.ts) defines the schema supporting custom endpoints: ```typescript table.enum('platform', ['groq', 'mistral', 'openrouter', 'custom']).notNullable(); table.text('base_url').nullable(); // For custom OpenAI-compatible endpoints ``` The `base_url` column is nullable for built-in providers but required for custom providers. The `Platform` enum in [`shared/types.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/shared/types.ts) includes `CUSTOM = 'custom'`, ensuring type safety across the codebase. --- ## Summary - **Single POST to `/keys/custom`** registers any OpenAI-compatible endpoint with FreeLLMAPI - **Dynamic provider instantiation** means no server restarts or configuration file edits required - **Automatic routing** via `resolveProvider()` in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) handles request forwarding - **Flexible authentication** supports both API-key and keyless services through the `OpenAICompatProvider.authHeader()` method - **Extended timeouts** (30 seconds) accommodate slower self-hosted inference compared to cloud providers --- ## Frequently Asked Questions ### Can I register multiple custom endpoints? Yes. Each unique `baseUrl` creates a distinct entry in the `api_keys` table with its own `key_id`. You can register the same model name on different endpoints—FreeLLMAPI routes based on the specific model-to-key mapping stored in the database. ### What if my custom endpoint uses a different path structure? The `OpenAICompatProvider` expects standard OpenAI API paths (`/v1/chat/completions`, `/v1/models`). If your service uses different paths, you may need to proxy it or modify the `baseUrl` to include any prefix paths before the standard `/v1/` segment. ### How does FreeLLMAPI handle authentication for local services? When you register with `"apiKey": "no-key"` or omit the field entirely, the provider's `authHeader()` method returns an empty object, and requests are sent without an `Authorization` header. This matches the behavior of local inference servers like Ollama or text-generation-inference in unauthenticated mode. ### Can I update an existing custom endpoint's models? Yes. Sending another `POST /keys/custom` request with the same `baseUrl` updates the existing `api_keys` row and adds or updates models in the `models` table. The handler in [`server/src/routes/keys.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/keys.ts) checks for existing entries and performs updates rather than duplicate inserts.