How the OpenAI-Compatible Adapter Works in FreeLLMAPI

The OpenAI-compatible adapter in FreeLLMAPI is implemented by the OpenAICompatProvider class in server/src/providers/openai-compat.ts, which intercepts requests targeting OpenAI API endpoints, translates them into internal formats, and normalizes diverse backend responses back into the standard OpenAI schema.

The FreeLLMAPI project provides a unified gateway for Large Language Model providers, and its OpenAI-compatible adapter enables seamless integration with existing OpenAI client libraries. This adapter allows the server to accept standard OpenAI API requests at endpoints like /v1/chat/completions and /v1/embeddings, forwarding them to various backend providers without requiring client-side code changes.

Core Architecture of the Adapter

The adapter centers on the OpenAICompatProvider class located in server/src/providers/openai-compat.ts. This provider registers under the platform identifier "openai-compat" within the provider registry defined in server/src/providers/index.ts, appearing alongside native providers such as groq, cloudflare, and google.

Request Normalization and Model Resolution

When a request hits an OpenAI-compatible endpoint, the adapter performs several transformation steps:

• Payload Parsing: The adapter converts OpenAI-style fields—including messages, max_tokens, and temperature—into the internal request model used by FreeLLMAPI.
• Model Discovery: The adapter resolves requested model identifiers (e.g., gpt-4o-mini or openai/gpt-4.1) via server/src/services/model-discovery.ts, which imports OpenAICompatProvider to discover compatible models across all connected platforms.
• Provider Routing: After resolution, the adapter constructs provider-specific HTTP requests using the generic client in server/src/services/router.ts, translating endpoint URLs, headers, and body formats to match the target backend's requirements.

Response Translation and Error Handling

Once the backend provider returns a result, the OpenAICompatProvider normalizes the raw response into the OpenAI response schema. This ensures the output contains standard fields such as id, object, choices, and usage. The adapter catches errors from underlying providers and re-wraps them into OpenAI-compatible error objects, integrating with the global rate-limiting and health-check infrastructure described in docs/architecture.md.

Supported Endpoints and Protocols

According to docs/api.md, the adapter exposes standard OpenAI API paths:

• /v1/chat/completions for chat-based inference
• /v1/completions for legacy text completion
• /v1/embeddings for vector generation

This modular approach allows FreeLLMAPI to maintain a drop-in replacement layer for OpenAI services while leveraging diverse LLM providers behind the scenes.

Practical Usage Examples

You can interact with the adapter using standard HTTP clients or the FreeLLMAPI CLI. The following examples demonstrate how the OpenAICompatProvider processes requests without requiring changes to existing OpenAI integration code.

Chat Completions via cURL

Send a chat completion request to the OpenAI-compatible endpoint:

curl https://api.your-freellmapi-instance.com/v1/chat/completions \
  -H "Authorization: Bearer $FREELLMAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Explain quantum tunnelling"}],
        "max_tokens": 512,
        "temperature": 0.7
      }'

The request is received by the OpenAI-compatible router and processed by OpenAICompatProvider, which forwards it to the best-available backend (such as Groq, OpenRouter, or self-hosted models) and returns a response matching the official OpenAI schema.

Embeddings Requests

For embedding operations, use the /v1/embeddings endpoint:

curl https://api.your-freellmapi-instance.com/v1/embeddings \
  -H "Authorization: Bearer $FREELLMAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "text-embedding-3-large",
        "input": ["Machine learning is a subset of AI."]
      }'

The adapter forwards this call to underlying providers (e.g., Google Vertex AI or local models) and returns vectors in the OpenAI-compatible format.

CLI Integration

You can also invoke the adapter through the FreeLLMAPI command-line interface:

freellmapi chat \
  --provider openai-compat \
  --model gpt-4o-mini \
  "Summarise the plot of Inception in 3 sentences."

The CLI translates arguments into OpenAI-style payloads and routes them through OpenAICompatProvider.

Summary

• The OpenAI-compatible adapter is implemented by the OpenAICompatProvider class in server/src/providers/openai-compat.ts.
• It registers as the "openai-compat" platform in server/src/providers/index.ts alongside providers like groq and cloudflare.
• The adapter normalizes OpenAI API requests into internal formats using server/src/services/model-discovery.ts for model resolution.
• Responses are translated back to the standard OpenAI schema (including choices, usage, and id fields) by the same provider class.
• Standard endpoints such as /v1/chat/completions and /v1/embeddings route through server/src/services/router.ts to reach diverse backend providers.

Frequently Asked Questions

What file contains the main OpenAI-compatible adapter implementation?

The core implementation resides in server/src/providers/openai-compat.ts within the OpenAICompatProvider class. This file handles request parsing, provider selection, and response normalization into the OpenAI schema.

How does FreeLLMAPI resolve models when using the OpenAI-compatible adapter?

Model resolution occurs in server/src/services/model-discovery.ts, which imports OpenAICompatProvider to map OpenAI-style model identifiers (like gpt-4o-mini) to available backends across all configured platforms.

Can existing OpenAI client libraries connect directly to FreeLLMAPI?

Yes. Because the adapter exposes standard endpoints such as /v1/chat/completions and returns identical response schemas—including id, object, choices, and usage fields—you can point any standard OpenAI client library to your FreeLLMAPI instance by changing only the base URL and API key.

How does the adapter handle errors from backend providers?

The OpenAICompatProvider catches errors from underlying providers and re-wraps them into OpenAI-compatible error objects. This integration ensures that client applications receive standardized error responses while the adapter participates in the global rate-limit and health-check infrastructure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}