# How to Integrate Custom OpenAI-Compatible Endpoints with FreeLLMAPI

> Integrate custom OpenAI compatible endpoints like llama.cpp and LM Studio with FreeLLMAPI. Register local services easily using the POST /custom endpoint for seamless API access.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-25

---

**FreeLLMAPI treats any OpenAI-compatible service as a "custom" provider, allowing developers to register local endpoints like llama.cpp or LM Studio by supplying a base URL and optional headers through the `POST /custom` endpoint.**

The `tashfeenahmed/freellmapi` platform provides native support for third-party OpenAI-compatible servers without requiring core code modifications. By leveraging the `OpenAICompatProvider` class and the custom key registration system, developers can route LLM requests through local instances of llama.cpp, LM Studio, Ollama, or self-hosted Cloudflare Workers endpoints. This architecture enables seamless integration of private or specialized models while maintaining the standard OpenAI Chat Completion API interface.

## Understanding the Custom Provider Architecture

FreeLLMAPI abstracts OpenAI-compatible endpoints through a unified provider system located in `server/src/providers/`. When you register a custom endpoint, the platform creates a dynamic `OpenAICompatProvider` instance bound specifically to your supplied base URL.

The **`OpenAICompatProvider`** class in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) handles the protocol translation between FreeLLMAPI and your endpoint. Its constructor (lines 21–36) stores the `baseUrl` and optional headers, while the `chatCompletion` method (lines 101–112) forwards POST requests to `${baseUrl}/chat/completions`. For streaming responses, the provider manages SSE connections through lines 166–188, automatically handling `stream: true` parameters.

If a custom endpoint returns non-JSON data, the provider throws a descriptive error (lines 55–60) directing developers to verify the correct `/v1` path suffix, ensuring proper URL configuration.

## Registering a Custom Endpoint

### Using the REST API

Developers register custom endpoints by calling the `POST /keys/custom` endpoint defined in [`server/src/routes/keys.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/keys.ts). This creates a unique API key row associated with your specific base URL.

```bash
curl -X POST https://api.freellmapi.com/keys/custom \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{
        "name": "Local LM Studio",
        "base_url": "http://127.0.0.1:8080/v1",
        "extra_headers": {}
      }'

```

The response includes a new `api_key` that routes requests to your specified endpoint. You can create multiple custom keys for different endpoints—each generates a distinct row in the database with `platform = 'custom'`.

### Database Registration Process

The registration flow in [`server/src/routes/keys.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/keys.ts) validates the request against `customProviderSchema` before inserting a record containing the `base_url` and platform identifier. One row is created per distinct URL, enabling several custom endpoints to coexist simultaneously. The system stores optional `extra_headers` for authentication tokens or custom HTTP headers required by private instances.

## Runtime Request Flow

When a chat completion request arrives, FreeLLMAPI resolves the appropriate provider through the `resolveProvider()` function in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts). For custom keys, this function instantiates a fresh `OpenAICompatProvider` configured with the specific `base_url` from the database.

The routing logic follows this path:

1. **Request received** at `/v1/chat/completions` with your custom API key
2. **Router selects** the custom key from the database ([`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts))
3. **Provider resolution** calls `resolveProvider('custom', key.base_url)` to build a provider instance
4. **Request forwarding** to your endpoint at `${baseUrl}/chat/completions`
5. **Response streaming** back to the original caller unchanged

The `OpenAICompatProvider` includes tooling-specific helpers for parallel tool calls and inline tool call rescue, ensuring compatibility with advanced OpenAI features even when using local models.

## Supported OpenAI-Compatible Servers

FreeLLMAPI works with any server implementing the OpenAI Chat Completion API format. Common integrations include:

- **LM Studio** – Use `http://127.0.0.1:8080/v1` when running the local server
- **Ollama** – Point to `http://localhost:11434/v1` with the OpenAI compatibility layer enabled
- **llama.cpp** – Configure the `--api-key` and `--host` parameters to expose the OpenAI-compatible endpoint
- **Cloudflare Workers AI** – Self-hosted instances with OpenAI-compatible routing

Each service requires only the base URL and optional authentication headers to integrate with FreeLLMAPI.

## Adding Custom Providers in Source Code

For Docker-based deployments or permanent integrations, developers can embed new providers directly into the source code by extending the `OpenAICompatProvider` class.

Create a new file in `server/src/providers/`:

```typescript
// server/src/providers/my-custom.ts
import { OpenAICompatProvider } from './openai-compat.js';

export const MyCustomProvider = new OpenAICompatProvider({
  platform: 'my-custom',
  name: 'My Local Service',
  baseUrl: 'http://host:1234/v1',
  extraHeaders: { Authorization: 'Bearer secret' },
});

```

Then register the provider in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts) by adding `MyCustomProvider` to the `allProviders` map. The router, quota system, and catalog sync will automatically recognize the new platform.

## Model Catalog Management

Custom endpoint models are handled separately from public providers. When you create a custom key, the system adds a model row with `platform = 'custom'` and `model_id` matching the remote model name.

The catalog synchronization service in [`server/src/services/catalog-sync.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/catalog-sync.ts) specifically skips records where `platform = 'custom'` when publishing to the public catalog. This keeps your local endpoint configurations private while maintaining them in the local database for routing purposes.

## Summary

- **Custom provider registration** occurs through `POST /keys/custom` in [`server/src/routes/keys.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/keys.ts), storing the `base_url` with platform type `custom`
- **Provider resolution** happens dynamically via `resolveProvider()` in [`server/src/providers/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/index.ts), creating `OpenAICompatProvider` instances per unique URL
- **Request forwarding** is handled by `OpenAICompatProvider.chatCompletion()` in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts), targeting `${baseUrl}/chat/completions`
- **Error handling** includes explicit validation for `/v1` path suffixes when endpoints return non-JSON responses
- **Model isolation** preserves custom endpoints locally while excluding them from public catalog sync in [`server/src/services/catalog-sync.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/catalog-sync.ts)

## Frequently Asked Questions

### What base URL format should I use for LM Studio?

Use `http://127.0.0.1:8080/v1` when LM Studio is running locally with default settings. The `/v1` suffix is mandatory—FreeLLMAPI will guide you with an error message if the endpoint returns non-JSON data, typically indicating a missing version path. Ensure the LM Studio server is running and the "OpenAI API" server option is enabled in the application settings.

### Can I add authentication headers to custom endpoints?

Yes, include `extra_headers` in the registration JSON when calling `POST /keys/custom`. These headers are stored in the database and passed through to your endpoint on every request. This supports Bearer tokens, API keys, or custom corporate headers required by private instances. The headers are injected by the `OpenAICompatProvider` constructor and sent with every chat completion request.

### How does FreeLLMAPI handle streaming responses from custom providers?

The `OpenAICompatProvider` in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts) handles streaming through dedicated logic in lines 166–188. When `stream: true` is specified in the request, the provider opens an SSE connection to your custom endpoint and forwards chunks back to the client unchanged. This maintains compatibility with OpenAI's streaming format while supporting local servers like llama.cpp or Ollama that implement server-sent events.

### Where are custom provider models stored in the catalog?

Custom models are stored locally in the database with `platform = 'custom'` but are excluded from public catalog synchronization. The [`server/src/services/catalog-sync.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/catalog-sync.ts) service explicitly skips these records when publishing to the shared model catalog, keeping your private endpoint configurations internal to your deployment while still making them available for routing decisions.