How to Integrate a Custom OpenAI-Compatible Endpoint with FreeLLMAPI

To integrate a custom OpenAI-compatible endpoint with FreeLLMAPI, register the base URL and credentials via the CLI, which stores them in the SQLite api_keys table and automatically discovers available models through the /models endpoint.

FreeLLMAPI functions as a router-plus-proxy that aggregates any service exposing an OpenAI-compatible HTTP API. Whether you are connecting a self-hosted model, a third-party gateway, or a mock server, the system treats your custom endpoint as a generic provider through the OpenAICompatProvider class.

How the OpenAI-Compatible Router Works

FreeLLMAPI delegates all request handling—chat completions, streaming, tool calls, and vision—to a unified provider abstraction. This architecture allows any endpoint mirroring the OpenAI API specification to integrate seamlessly.

The OpenAICompatProvider Class

The core integration logic resides in server/src/providers/openai-compat.ts. The OpenAICompatProvider class handles request building, streaming transformations, and error recovery. When routing requests, the provider constructs the appropriate HTTP headers, manages bearer token injection, and forwards payloads unchanged to the upstream endpoint.

Per-Endpoint Credential Storage

Unlike traditional API gateways that bind credentials to specific models, FreeLLMAPI stores credentials per endpoint URL. This design, implemented in server/src/services/custom-endpoint.ts, enables multi-tenant scenarios where a single base URL hosts multiple API keys. The resolveCustomEndpointKey function manages this by encrypting keys via the encrypt utility and persisting them to the SQLite api_keys table.

Step-by-Step Registration Process

Integrating a custom endpoint requires four distinct phases: registration, credential encryption, model discovery, and request routing.

1. Register via CLI

Use the freellmapi register command to add your endpoint. The CLI transmits a POST request to /api/custom-endpoints with your base_url, optional api_key, and label.

freellmapi register \
  --base-url https://my-local-model.example.com/v1 \
  --api-key sk-mysecretkey \
  --label "My Local Model"

Behind the scenes, the resolveCustomEndpointKey function in server/src/services/custom-endpoint.ts either inserts a new row or updates an existing placeholder (no-key) entry, enabling multiple credentials per URL.

2. Encrypt and Store Credentials

The API key undergoes encryption using the internal encrypt function before storage. If the endpoint already exists in the api_keys table, the touch method updates the row in-place; otherwise, insertKey creates a new encrypted entry.

3. Automatic Model Discovery

Upon the first request requiring model selection, the router invokes provider.fetchModelCatalog(). This method performs a GET ${base_url}/models request to your endpoint, retrieves the raw JSON response, and transforms it into CatalogModel objects. The discovered models are cached and exposed through FreeLLMAPI's own /v1/models endpoint.

curl -s -H "Authorization: Bearer $FREELLMAPI_API_KEY" \
  http://localhost:3000/v1/models | jq .

4. Request Routing

When a client sends a request to http://localhost:3000/v1/chat/completions, the router selects the appropriate model (either auto-selected or specified via the model parameter) and forwards the request to your custom endpoint. The chatCompletion and streamChatCompletion methods in OpenAICompatProvider handle the proxying, injecting the stored credential as a bearer token unless the provider operates in keyless mode.

Configuration Examples

Registering with Authentication

For endpoints requiring API keys, include the --api-key flag during registration. The credential is encrypted and associated with the base URL.

freellmapi register \
  --base-url https://api.thirdparty.com/v1 \
  --api-key sk-abc123 \
  --label "Third-Party Gateway"

Configuring Keyless Endpoints

Some providers, such as local inference servers or Kilo's free tier, do not require authentication. Register these using the --keyless flag, which sets the keyless flag in the OpenAICompatProvider constructor and omits the Authorization header from upstream requests.

freellmapi register \
  --base-url https://my-anonymous-gateway.example.com/v1 \
  --keyless \
  --label "Anonymous Gateway"

Verifying the Integration

Test your custom endpoint by sending a standard OpenAI-format request through the FreeLLMAPI proxy:

curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer $FREELLMAPI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "openai/gpt-4o",
        "messages": [{"role":"user","content":"Hello, world!"}]
      }' | jq .

All standard OpenAI parameters—including stream, tools, and max_tokens—pass through transparently to your backend.

Client Integration

Using Standard OpenAI SDKs

Point any OpenAI-compatible client to http://localhost:3000/v1 (or your configured gateway URL). The openapiSpec defined in server/src/docs/openapi.ts ensures compatibility with standard client implementations.

Generating IDE Configuration Files

FreeLLMAPI includes convenience generators in cli/src/tools.ts that emit ready-made configuration files for popular IDE assistants. The getTool function retrieves generators for Claude Code, Codex, Cline, and others, each producing environment-specific config files.

import { getTool } from './cli/src/tools.js';

const ctx = {
  url: 'http://localhost:3000',
  apiKey: process.env.FREELLMAPI_API_KEY,
  homeDir: process.env.HOME,
  profile: 'default',
  models: [{ id: 'openai/gpt-4o', context_window: 128_000 }],
  requestedModelId: undefined,
};

const claudeGen = getTool('claude')?.generate(ctx);
console.log(claudeGen?.files[0].path);   // → ~/.claude/settings.json

After generation, tools like Codex automatically route requests through FreeLLMAPI:

freellmapi tools codex
codex --profile default

Advanced Provider Features

Parallel Tool Call Handling

Certain backends, such as NVIDIA NIM, reject parallel tool invocations. The resolveParallelToolCalls logic in OpenAICompatProvider automatically forces parallel_tool_calls: false when the forceSingleToolCall flag is set, ensuring compatibility with restrictive endpoints.

Error Rescue Mechanisms

If an upstream provider returns a tool-call failure with raw text instead of structured JSON—common with Groq and similar services—the provider's rescueFailedGeneration method intercepts the error and reformats it into a valid OpenAI-compatible response, preventing client-side failures.

Summary

• Register endpoints using freellmapi register to store encrypted credentials in the api_keys table via resolveCustomEndpointKey.
• Support multiple keys per URL through the per-endpoint credential model implemented in server/src/services/custom-endpoint.ts.
• Discover models automatically via the fetchModelCatalog method, which queries ${BASE_URL}/models and caches the results.
• Handle keyless providers by passing the --keyless flag, which disables bearer token injection in OpenAICompatProvider.
• Generate client configs using the tools module in cli/src/tools.ts for instant integration with Claude, Codex, and Cline.

Frequently Asked Questions

Can I use multiple API keys with the same custom endpoint URL?

Yes. FreeLLMAPI stores credentials per-endpoint rather than per-model, allowing you to register multiple keys for the same base URL. The resolveCustomEndpointKey function in server/src/services/custom-endpoint.ts manages this by updating existing rows or inserting new ones, enabling multi-tenant scenarios where different users access the same backend through different credentials.

How does FreeLLMAPI handle custom endpoints that do not require authentication?

Set the --keyless flag during registration. This configures the OpenAICompatProvider class to omit the Authorization header entirely when forwarding requests, as implemented in the provider's authHeader logic. This is essential for local inference servers or public gateways that reject authenticated requests.

What happens if my custom endpoint returns models in a non-standard format?

The fetchModelCatalog method in server/src/providers/openai-compat.ts retrieves the raw /models response and passes it to the router's model-selection logic without strict parsing. As long as your endpoint returns a JSON list containing model identifiers, FreeLLMAPI will surface them through its own /v1/models endpoint, regardless of additional custom fields.

Can I integrate custom endpoints with VS Code extensions or other IDEs?

Yes. Use the tools generators in cli/src/tools.ts to create configuration files for specific IDE assistants. Each generator (e.g., getTool('claude'), getTool('codex')) produces the necessary environment variables and config files to redirect the IDE's AI features to your FreeLLMAPI gateway, which then routes to your custom endpoint.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}