How to Configure Custom OpenAI-Compatible Endpoints in FreeLLMAPI: A Complete Guide

You can configure custom OpenAI-compatible endpoints in FreeLLMAPI by sending a POST request to /keys/custom with the baseUrl and model identifiers, which stores the configuration in the database and automatically routes requests through the OpenAICompatProvider class.

FreeLLMAPI treats any OpenAI-compatible service as a provider, allowing you to integrate self-hosted inference servers, local LLMs like Ollama, or third-party APIs alongside built-in providers such as Groq and Mistral. This architecture enables dynamic registration of custom endpoints without modifying the core codebase, routing all chat completion requests through the same standardized interface.

Architecture Overview

FreeLLMAPI implements custom endpoints through a dynamic provider resolution system. Unlike built-in providers that are instantiated once at startup in server/src/providers/index.ts, custom providers are created on-the-fly for each distinct base URL you register.

Database Storage and Model Registration

When you register a custom endpoint, the system creates two database entries:

1. API Key Entry: A row is inserted into the api_keys table with platform = 'custom' and the supplied base_url (as defined in server/src/db/migrations.ts).
2. Model Registration: Each model listed in your request is inserted into the models table with platform = 'custom' and a foreign key (key_id) pointing to the newly created API key row.

Runtime Provider Resolution

When a request targets a model belonging to the custom platform, the resolveProvider() function in server/src/providers/index.ts builds a fresh OpenAICompatProvider instance using the stored baseUrl and an extended timeout configuration for self-hosted inference.

Registering a Custom Endpoint

To add a custom OpenAI-compatible endpoint, send a POST request to the /keys/custom route handled by keysRouter in server/src/routes/keys.ts.

Required Parameters

The request body must contain:

• baseUrl: The OpenAI-compatible endpoint URL (e.g., http://localhost:11434/v1)
• models: An array of model identifiers (strings or objects with model and displayName properties)
• apiKey (optional): Authentication token for the endpoint, or "no-key" for keyless services
• label (optional): Human-readable identifier for the endpoint

Registration Example

curl -X POST https://your-llmapi.example.com/keys/custom \
  -H "Content-Type: application/json" \
  -d '{
    "baseUrl": "http://localhost:11434/v1",
    "apiKey": "my-ollama-key",
    "label": "Local Ollama",
    "models": [
      {"model": "llama3:8b", "displayName": "Llama 3 8B"},
      "phi2"
    ]
  }'

The keysRouter.post('/custom', ...) handler parses this payload, creates or updates the api_keys row, and registers each model in the models table (see server/src/routes/keys.ts, lines 47-75 and 94-115).

Response structure:

{
  "success": true,
  "keyId": 23,
  "platform": "custom",
  "baseUrl": "http://localhost:11434/v1",
  "models": [
    {"modelDbId": 101, "model": "llama3:8b", "displayName": "Llama 3 8B"},
    {"modelDbId": 102, "model": "phi2", "displayName": "phi2"}
  ],
  "maskedKey": "****"
}

Provider Resolution and Runtime

Once registered, custom endpoints are automatically resolved at request time without additional configuration.

The resolveProvider() Function

When a client requests a completion using a custom model, the system calls resolveProvider() in server/src/providers/index.ts:

// server/src/providers/index.ts
export function resolveProvider(platform: Platform, baseUrl?: string | null): BaseProvider | undefined {
  if (platform === 'custom') {
    const trimmed = baseUrl?.trim();
    if (!trimmed) return undefined;
    return new OpenAICompatProvider({
      platform: 'custom',
      name: 'Custom (OpenAI-compatible)',
      baseUrl: trimmed,
      timeoutMs: CUSTOM_PROVIDER_TIMEOUT_MS,
    });
  }
  return providers.get(platform);
}

This function instantiates an OpenAICompatProvider with the stored baseUrl and a custom timeout suitable for self-hosted inference.

OpenAICompatProvider Implementation

The OpenAICompatProvider class, implemented in server/src/providers/openai-compat.ts, forwards chat completion calls to baseUrl + '/chat/completions'. It applies the stored API key via its authHeader() method (lines 87-92), or omits authentication for keyless services configured with "no-key".

Usage Examples

After registration, use custom models exactly like built-in providers through the standard OpenAI-compatible API interface.

Chat Completions

curl -X POST https://your-llmapi.example.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3:8b",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

The server looks up the model in the database, finds the associated platform = "custom" and key_id, resolves the provider using resolveProvider(), and forwards the request to http://localhost:11434/v1/chat/completions with the appropriate authentication headers.

Summary

• Register endpoints via POST /keys/custom with baseUrl and model identifiers in server/src/routes/keys.ts.
• Store configuration in the api_keys and models tables with platform = 'custom' as defined in server/src/db/migrations.ts.
• Automatic resolution occurs through resolveProvider() in server/src/providers/index.ts, which instantiates OpenAICompatProvider for custom platforms.
• Runtime forwarding sends requests to baseUrl + '/chat/completions' using the OpenAICompatProvider class in server/src/providers/openai-compat.ts.

Frequently Asked Questions

What parameters are required to register a custom endpoint?

You must provide a baseUrl pointing to the OpenAI-compatible API endpoint and at least one model identifier in the models array. The apiKey parameter is optional—omit it or set it to "no-key" for keyless services like local Ollama instances.

How does FreeLLMAPI handle authentication for custom endpoints?

The system stores the provided apiKey in the api_keys table and retrieves it via the authHeader() method in OpenAICompatProvider (server/src/providers/openai-compat.ts). For keyless services, the authentication header is omitted entirely when "no-key" is specified during registration.

Can I register multiple models from the same custom endpoint?

Yes. The models array in the registration request accepts multiple entries, allowing you to register all available models from a single base URL. Each model creates a separate row in the models table linked to the same api_keys row via the key_id foreign key.

Where is the custom endpoint configuration stored?

Configuration persists in two SQLite tables: api_keys stores the base_url and credentials (added via migrations in server/src/db/migrations.ts), while models stores individual model identifiers with platform = 'custom' and a reference to the parent key record.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}