How to Integrate Custom OpenAI-Compatible Endpoints with FreeLLMAPI

FreeLLMAPI treats any OpenAI-compatible service as a "custom" provider, allowing developers to register local endpoints like llama.cpp or LM Studio by supplying a base URL and optional headers through the POST /custom endpoint.

The tashfeenahmed/freellmapi platform provides native support for third-party OpenAI-compatible servers without requiring core code modifications. By leveraging the OpenAICompatProvider class and the custom key registration system, developers can route LLM requests through local instances of llama.cpp, LM Studio, Ollama, or self-hosted Cloudflare Workers endpoints. This architecture enables seamless integration of private or specialized models while maintaining the standard OpenAI Chat Completion API interface.

Understanding the Custom Provider Architecture

FreeLLMAPI abstracts OpenAI-compatible endpoints through a unified provider system located in server/src/providers/. When you register a custom endpoint, the platform creates a dynamic OpenAICompatProvider instance bound specifically to your supplied base URL.

The OpenAICompatProvider class in server/src/providers/openai-compat.ts handles the protocol translation between FreeLLMAPI and your endpoint. Its constructor (lines 21–36) stores the baseUrl and optional headers, while the chatCompletion method (lines 101–112) forwards POST requests to ${baseUrl}/chat/completions. For streaming responses, the provider manages SSE connections through lines 166–188, automatically handling stream: true parameters.

If a custom endpoint returns non-JSON data, the provider throws a descriptive error (lines 55–60) directing developers to verify the correct /v1 path suffix, ensuring proper URL configuration.

Registering a Custom Endpoint

Using the REST API

Developers register custom endpoints by calling the POST /keys/custom endpoint defined in server/src/routes/keys.ts. This creates a unique API key row associated with your specific base URL.

curl -X POST https://api.freellmapi.com/keys/custom \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{
        "name": "Local LM Studio",
        "base_url": "http://127.0.0.1:8080/v1",
        "extra_headers": {}
      }'

The response includes a new api_key that routes requests to your specified endpoint. You can create multiple custom keys for different endpoints—each generates a distinct row in the database with platform = 'custom'.

Database Registration Process

The registration flow in server/src/routes/keys.ts validates the request against customProviderSchema before inserting a record containing the base_url and platform identifier. One row is created per distinct URL, enabling several custom endpoints to coexist simultaneously. The system stores optional extra_headers for authentication tokens or custom HTTP headers required by private instances.

Runtime Request Flow

When a chat completion request arrives, FreeLLMAPI resolves the appropriate provider through the resolveProvider() function in server/src/providers/index.ts. For custom keys, this function instantiates a fresh OpenAICompatProvider configured with the specific base_url from the database.

The routing logic follows this path:

1. Request received at /v1/chat/completions with your custom API key
2. Router selects the custom key from the database (server/src/services/router.ts)
3. Provider resolution calls resolveProvider('custom', key.base_url) to build a provider instance
4. Request forwarding to your endpoint at ${baseUrl}/chat/completions
5. Response streaming back to the original caller unchanged

The OpenAICompatProvider includes tooling-specific helpers for parallel tool calls and inline tool call rescue, ensuring compatibility with advanced OpenAI features even when using local models.

Supported OpenAI-Compatible Servers

FreeLLMAPI works with any server implementing the OpenAI Chat Completion API format. Common integrations include:

• LM Studio – Use http://127.0.0.1:8080/v1 when running the local server
• Ollama – Point to http://localhost:11434/v1 with the OpenAI compatibility layer enabled
• llama.cpp – Configure the --api-key and --host parameters to expose the OpenAI-compatible endpoint
• Cloudflare Workers AI – Self-hosted instances with OpenAI-compatible routing

Each service requires only the base URL and optional authentication headers to integrate with FreeLLMAPI.

Adding Custom Providers in Source Code

For Docker-based deployments or permanent integrations, developers can embed new providers directly into the source code by extending the OpenAICompatProvider class.

Create a new file in server/src/providers/:

// server/src/providers/my-custom.ts
import { OpenAICompatProvider } from './openai-compat.js';

export const MyCustomProvider = new OpenAICompatProvider({
  platform: 'my-custom',
  name: 'My Local Service',
  baseUrl: 'http://host:1234/v1',
  extraHeaders: { Authorization: 'Bearer secret' },
});

Then register the provider in server/src/providers/index.ts by adding MyCustomProvider to the allProviders map. The router, quota system, and catalog sync will automatically recognize the new platform.

Model Catalog Management

Custom endpoint models are handled separately from public providers. When you create a custom key, the system adds a model row with platform = 'custom' and model_id matching the remote model name.

The catalog synchronization service in server/src/services/catalog-sync.ts specifically skips records where platform = 'custom' when publishing to the public catalog. This keeps your local endpoint configurations private while maintaining them in the local database for routing purposes.

Summary

• Custom provider registration occurs through POST /keys/custom in server/src/routes/keys.ts, storing the base_url with platform type custom
• Provider resolution happens dynamically via resolveProvider() in server/src/providers/index.ts, creating OpenAICompatProvider instances per unique URL
• Request forwarding is handled by OpenAICompatProvider.chatCompletion() in server/src/providers/openai-compat.ts, targeting ${baseUrl}/chat/completions
• Error handling includes explicit validation for /v1 path suffixes when endpoints return non-JSON responses
• Model isolation preserves custom endpoints locally while excluding them from public catalog sync in server/src/services/catalog-sync.ts

Frequently Asked Questions

What base URL format should I use for LM Studio?

Use http://127.0.0.1:8080/v1 when LM Studio is running locally with default settings. The /v1 suffix is mandatory—FreeLLMAPI will guide you with an error message if the endpoint returns non-JSON data, typically indicating a missing version path. Ensure the LM Studio server is running and the "OpenAI API" server option is enabled in the application settings.

Can I add authentication headers to custom endpoints?

Yes, include extra_headers in the registration JSON when calling POST /keys/custom. These headers are stored in the database and passed through to your endpoint on every request. This supports Bearer tokens, API keys, or custom corporate headers required by private instances. The headers are injected by the OpenAICompatProvider constructor and sent with every chat completion request.

How does FreeLLMAPI handle streaming responses from custom providers?

The OpenAICompatProvider in server/src/providers/openai-compat.ts handles streaming through dedicated logic in lines 166–188. When stream: true is specified in the request, the provider opens an SSE connection to your custom endpoint and forwards chunks back to the client unchanged. This maintains compatibility with OpenAI's streaming format while supporting local servers like llama.cpp or Ollama that implement server-sent events.

Where are custom provider models stored in the catalog?

Custom models are stored locally in the database with platform = 'custom' but are excluded from public catalog synchronization. The server/src/services/catalog-sync.ts service explicitly skips these records when publishing to the shared model catalog, keeping your private endpoint configurations internal to your deployment while still making them available for routing decisions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}