# How to Integrate a Custom OpenAI-Compatible Endpoint with FreeLLMAPI

> Easily integrate your custom OpenAI-compatible endpoint with FreeLLMAPI. Register your URL and credentials via CLI to connect and discover models seamlessly.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-26

---

**To integrate a custom OpenAI-compatible endpoint with FreeLLMAPI, register the base URL and credentials via the CLI, which stores them in the SQLite `api_keys` table and automatically discovers available models through the `/models` endpoint.**

FreeLLMAPI functions as a **router-plus-proxy** that aggregates any service exposing an OpenAI-compatible HTTP API. Whether you are connecting a self-hosted model, a third-party gateway, or a mock server, the system treats your custom endpoint as a generic provider through the `OpenAICompatProvider` class.

## How the OpenAI-Compatible Router Works

FreeLLMAPI delegates all request handling—chat completions, streaming, tool calls, and vision—to a unified provider abstraction. This architecture allows any endpoint mirroring the OpenAI API specification to integrate seamlessly.

### The OpenAICompatProvider Class

The core integration logic resides in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts). The `OpenAICompatProvider` class handles request building, streaming transformations, and error recovery. When routing requests, the provider constructs the appropriate HTTP headers, manages bearer token injection, and forwards payloads unchanged to the upstream endpoint.

### Per-Endpoint Credential Storage

Unlike traditional API gateways that bind credentials to specific models, FreeLLMAPI stores credentials per **endpoint URL**. This design, implemented in [`server/src/services/custom-endpoint.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/custom-endpoint.ts), enables multi-tenant scenarios where a single base URL hosts multiple API keys. The `resolveCustomEndpointKey` function manages this by encrypting keys via the `encrypt` utility and persisting them to the SQLite `api_keys` table.

## Step-by-Step Registration Process

Integrating a custom endpoint requires four distinct phases: registration, credential encryption, model discovery, and request routing.

### 1. Register via CLI

Use the `freellmapi register` command to add your endpoint. The CLI transmits a `POST` request to `/api/custom-endpoints` with your `base_url`, optional `api_key`, and label.

```bash
freellmapi register \
  --base-url https://my-local-model.example.com/v1 \
  --api-key sk-mysecretkey \
  --label "My Local Model"

```

Behind the scenes, the `resolveCustomEndpointKey` function in [`server/src/services/custom-endpoint.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/custom-endpoint.ts) either inserts a new row or updates an existing placeholder (`no-key`) entry, enabling multiple credentials per URL.

### 2. Encrypt and Store Credentials

The API key undergoes encryption using the internal `encrypt` function before storage. If the endpoint already exists in the `api_keys` table, the `touch` method updates the row in-place; otherwise, `insertKey` creates a new encrypted entry.

### 3. Automatic Model Discovery

Upon the first request requiring model selection, the router invokes `provider.fetchModelCatalog()`. This method performs a `GET ${base_url}/models` request to your endpoint, retrieves the raw JSON response, and transforms it into `CatalogModel` objects. The discovered models are cached and exposed through FreeLLMAPI's own `/v1/models` endpoint.

```bash
curl -s -H "Authorization: Bearer $FREELLMAPI_API_KEY" \
  http://localhost:3000/v1/models | jq .

```

### 4. Request Routing

When a client sends a request to `http://localhost:3000/v1/chat/completions`, the router selects the appropriate model (either auto-selected or specified via the `model` parameter) and forwards the request to your custom endpoint. The `chatCompletion` and `streamChatCompletion` methods in `OpenAICompatProvider` handle the proxying, injecting the stored credential as a bearer token unless the provider operates in keyless mode.

## Configuration Examples

### Registering with Authentication

For endpoints requiring API keys, include the `--api-key` flag during registration. The credential is encrypted and associated with the base URL.

```bash
freellmapi register \
  --base-url https://api.thirdparty.com/v1 \
  --api-key sk-abc123 \
  --label "Third-Party Gateway"

```

### Configuring Keyless Endpoints

Some providers, such as local inference servers or Kilo's free tier, do not require authentication. Register these using the `--keyless` flag, which sets the `keyless` flag in the `OpenAICompatProvider` constructor and omits the `Authorization` header from upstream requests.

```bash
freellmapi register \
  --base-url https://my-anonymous-gateway.example.com/v1 \
  --keyless \
  --label "Anonymous Gateway"

```

### Verifying the Integration

Test your custom endpoint by sending a standard OpenAI-format request through the FreeLLMAPI proxy:

```bash
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer $FREELLMAPI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "openai/gpt-4o",
        "messages": [{"role":"user","content":"Hello, world!"}]
      }' | jq .

```

All standard OpenAI parameters—including `stream`, `tools`, and `max_tokens`—pass through transparently to your backend.

## Client Integration

### Using Standard OpenAI SDKs

Point any OpenAI-compatible client to `http://localhost:3000/v1` (or your configured gateway URL). The `openapiSpec` defined in [`server/src/docs/openapi.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/docs/openapi.ts) ensures compatibility with standard client implementations.

### Generating IDE Configuration Files

FreeLLMAPI includes convenience generators in [`cli/src/tools.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/tools.ts) that emit ready-made configuration files for popular IDE assistants. The `getTool` function retrieves generators for Claude Code, Codex, Cline, and others, each producing environment-specific config files.

```ts
import { getTool } from './cli/src/tools.js';

const ctx = {
  url: 'http://localhost:3000',
  apiKey: process.env.FREELLMAPI_API_KEY,
  homeDir: process.env.HOME,
  profile: 'default',
  models: [{ id: 'openai/gpt-4o', context_window: 128_000 }],
  requestedModelId: undefined,
};

const claudeGen = getTool('claude')?.generate(ctx);
console.log(claudeGen?.files[0].path);   // → ~/.claude/settings.json

```

After generation, tools like Codex automatically route requests through FreeLLMAPI:

```bash
freellmapi tools codex
codex --profile default

```

## Advanced Provider Features

### Parallel Tool Call Handling

Certain backends, such as NVIDIA NIM, reject parallel tool invocations. The `resolveParallelToolCalls` logic in `OpenAICompatProvider` automatically forces `parallel_tool_calls: false` when the `forceSingleToolCall` flag is set, ensuring compatibility with restrictive endpoints.

### Error Rescue Mechanisms

If an upstream provider returns a tool-call failure with raw text instead of structured JSON—common with Groq and similar services—the provider's `rescueFailedGeneration` method intercepts the error and reformats it into a valid OpenAI-compatible response, preventing client-side failures.

## Summary

- **Register endpoints** using `freellmapi register` to store encrypted credentials in the `api_keys` table via `resolveCustomEndpointKey`.
- **Support multiple keys per URL** through the per-endpoint credential model implemented in [`server/src/services/custom-endpoint.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/custom-endpoint.ts).
- **Discover models automatically** via the `fetchModelCatalog` method, which queries `${BASE_URL}/models` and caches the results.
- **Handle keyless providers** by passing the `--keyless` flag, which disables bearer token injection in `OpenAICompatProvider`.
- **Generate client configs** using the `tools` module in [`cli/src/tools.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/tools.ts) for instant integration with Claude, Codex, and Cline.

## Frequently Asked Questions

### Can I use multiple API keys with the same custom endpoint URL?

Yes. FreeLLMAPI stores credentials per-endpoint rather than per-model, allowing you to register multiple keys for the same base URL. The `resolveCustomEndpointKey` function in [`server/src/services/custom-endpoint.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/custom-endpoint.ts) manages this by updating existing rows or inserting new ones, enabling multi-tenant scenarios where different users access the same backend through different credentials.

### How does FreeLLMAPI handle custom endpoints that do not require authentication?

Set the `--keyless` flag during registration. This configures the `OpenAICompatProvider` class to omit the `Authorization` header entirely when forwarding requests, as implemented in the provider's `authHeader` logic. This is essential for local inference servers or public gateways that reject authenticated requests.

### What happens if my custom endpoint returns models in a non-standard format?

The `fetchModelCatalog` method in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) retrieves the raw `/models` response and passes it to the router's model-selection logic without strict parsing. As long as your endpoint returns a JSON list containing model identifiers, FreeLLMAPI will surface them through its own `/v1/models` endpoint, regardless of additional custom fields.

### Can I integrate custom endpoints with VS Code extensions or other IDEs?

Yes. Use the `tools` generators in [`cli/src/tools.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/tools.ts) to create configuration files for specific IDE assistants. Each generator (e.g., `getTool('claude')`, `getTool('codex')`) produces the necessary environment variables and config files to redirect the IDE's AI features to your FreeLLMAPI gateway, which then routes to your custom endpoint.