How to Add a Custom OpenAI-Compatible Endpoint to FreeLLMAPI

You can add a custom OpenAI-compatible endpoint to FreeLLMAPI by sending a POST request to /keys/custom with your baseUrl, optional apiKey, and model definitions, which the system registers as a dynamic provider that automatically routes chat completion requests to your specified endpoint.

FreeLLMAPI normalizes access to large language models by treating any OpenAI-compatible service as a provider. Unlike built-in providers that are hardcoded at startup, custom endpoints are instantiated dynamically per unique base URL. This architecture allows you to self-host models or integrate third-party APIs without modifying the core codebase.

Architecture Overview

Registration Flow

The registration process begins when you POST to /keys/custom handled by the keysRouter in server/src/routes/keys.ts. The system validates your baseUrl and creates a database entry in the api_keys table with platform = 'custom' and the supplied base_url (added via the migration in server/src/db/migrations.ts). Each model you specify is inserted into the models table with a foreign key referencing your API key row.

Provider Resolution at Runtime

When a client requests a model belonging to the custom platform (defined in shared/types.ts), resolveProvider() in server/src/providers/index.ts instantiates a fresh OpenAICompatProvider. This provider uses your stored baseUrl and an extended timeout (CUSTOM_PROVIDER_TIMEOUT_MS) to accommodate self-hosted inference. The OpenAICompatProvider class in server/src/providers/openai-compat.ts forwards requests to baseUrl + '/chat/completions' and injects the stored API key via its authHeader() method.

Step-by-Step Implementation

Registering Your Custom Endpoint

Send a POST request to the /keys/custom route with your endpoint configuration:

curl -X POST https://your-llmapi.example.com/keys/custom \
  -H "Content-Type: application/json" \
  -d '{
        "baseUrl": "http://localhost:11434/v1",
        "apiKey": "my-ollama-key",
        "label": "Local Ollama",
        "models": [
          {"model": "llama3:8b", "displayName": "Llama 3 8B"},
          "phi2"
        ]
      }'

The apiKey field is optional; use "no-key" for keyless services. The keysRouter.post('/custom', ...) handler parses this payload, creates the api_keys row, and registers each model in the models table with the appropriate foreign key relationships.

Provider Resolution Logic

When a request targets your registered model, FreeLLMAPI resolves the provider dynamically:

// server/src/providers/index.ts
export function resolveProvider(platform: Platform, baseUrl?: string | null): BaseProvider | undefined {
  if (platform === 'custom') {
    const trimmed = baseUrl?.trim();
    if (!trimmed) return undefined;
    return new OpenAICompatProvider({
      platform: 'custom',
      name: 'Custom (OpenAI-compatible)',
      baseUrl: trimmed,
      timeoutMs: CUSTOM_PROVIDER_TIMEOUT_MS,
    });
  }
  return providers.get(platform);
}

This function ensures that each custom endpoint receives its own isolated provider instance with the correct base URL and authentication credentials.

Making Requests to Your Custom Endpoint

Once registered, use standard OpenAI-compatible requests targeting your model:

curl -X POST https://your-llmapi.example.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "llama3:8b",
        "messages": [{"role":"user","content":"Hello"}]
      }'

The server looks up the model in the database, identifies it as platform = "custom", resolves the provider using the stored baseUrl, and forwards the request to http://localhost:11434/v1/chat/completions.

Key Source Files

Understanding these files helps you debug or extend custom endpoint functionality:

• server/src/routes/keys.ts: Contains the keysRouter.post('/custom', ...) handler that processes registration requests and manages database transactions for api_keys and models tables.
• server/src/providers/index.ts: Implements resolveProvider() which constructs OpenAICompatProvider instances for custom platforms.
• server/src/providers/openai-compat.ts: The generic adapter that translates FreeLLMAPI requests to OpenAI-compatible formats, handling paths like /chat/completions and API key injection.
• server/src/db/migrations.ts: Contains the schema migration that adds the base_url column to the api_keys table, enabling custom provider storage.
• shared/types.ts: Defines the Platform enum including the 'custom' type used throughout the provider resolution logic.

Summary

• FreeLLMAPI treats custom endpoints as dynamic providers created on-the-fly via the /keys/custom endpoint.
• Registration requires only a baseUrl and model list; the system handles database persistence in api_keys and models tables.
• Runtime resolution occurs through resolveProvider() in server/src/providers/index.ts, which instantiates OpenAICompatProvider with your specific configuration.
• The provider forwards requests to baseUrl + '/chat/completions' using the stored credentials, supporting both authenticated and keyless services.

Frequently Asked Questions

Do I need to restart FreeLLMAPI after adding a custom endpoint?

No. FreeLLMAPI creates custom providers dynamically at request time. Once you receive a successful response from POST /keys/custom, the endpoint is immediately available for chat completions without restarting the server.

Can I use custom endpoints without an API key?

Yes. Set the apiKey field to "no-key" when registering the endpoint. The OpenAICompatProvider will omit authentication headers when calling your base URL, making it compatible with keyless self-hosted models like Ollama running locally.

How does FreeLLMAPI handle timeouts for self-hosted custom endpoints?

Custom providers receive an extended timeout defined by CUSTOM_PROVIDER_TIMEOUT_MS in server/src/providers/index.ts. This accommodates slower inference speeds typical of self-hosted hardware compared to commercial APIs.

What happens if my custom endpoint base URL changes?

You must register a new custom endpoint with the updated baseUrl. FreeLLMAPI stores the base URL in the api_keys table at registration time and does not support dynamic updates to existing custom provider URLs; each unique base URL creates a distinct provider instance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}