How to Use the FreeLLMAPI /v1/chat/completions Endpoint: A Complete Guide

Use the OpenAI-compatible endpoint at http://localhost:3001/v1/chat/completions with a unified Bearer token to route chat requests through free-tier providers like Groq, Cohere, or Google Gemini using automatic model selection.

FreeLLMAPI acts as a unified gateway that transforms multiple free LLM providers into a single OpenAI-compatible interface. This article explains how to interact with the /v1/chat/completions endpoint based on the implementation in the tashfeenahmed/freellmapi repository, covering authentication, intelligent routing, streaming, and advanced features.

Endpoint Basics and Authentication

The chat completions endpoint follows the standard OpenAI API specification, ensuring compatibility with existing SDKs and third-party tools.

Base URL and Unified Key Authentication

All requests target:

http://localhost:3001/v1/chat/completions

The proxy implementation in server/src/routes/proxy.ts validates requests using unified key authentication. Send your key in the Authorization header:

Authorization: Bearer freellmapi-your-unified-key

The system maps this unified key to specific provider-level credentials configured through the dashboard's Keys page. This abstraction allows you to rotate underlying provider keys without changing client code.

Request Structure and Routing Logic

The request body follows the OpenAI Chat Completions schema, with FreeLLMAPI-specific routing extensions.

The Auto-Routing System

When you specify "model": "auto", the router activates the fallback chain algorithm implemented in server/src/lib/fallback-loop.ts. The system iterates through enabled free providers (Groq, Cohere, Gemini), attempting the request with each until one succeeds. This ensures resilient failover if a provider returns errors or rate limits.

Model Selection Strategies

Beyond simple auto, you can use prefixed profiles to control routing behavior:

• auto:fast - Prioritizes low-latency providers
• auto:smart - Selects higher-quality models
• auto:balanced - Default behavior balancing speed and quality

According to server/src/routes/proxy.ts, the router inspects these prefixes and selects the appropriate provider chain based on your configuration.

Advanced Capabilities

FreeLLMAPI supports the full OpenAI feature set including streaming, tool use, and vision inputs through the generic client in server/src/providers/openai-compat.ts.

Streaming Responses

Enable streaming by setting stream: true. The proxy forwards Server-Sent Events (SSE) chunks exactly as OpenAI does, preserving delta objects and the final finish_reason:

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream me a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Tool Calling and Function Execution

The endpoint supports OpenAI-style function calling. The proxy passes tools and tool_choice fields unchanged to the underlying provider, then re-wraps the response into standard OpenAI format.

Implement a two-turn tool execution:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

# Request 1: Get the tool call

first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required",
)

call = first.choices[0].message.tool_calls[0]

# Request 2: Submit the tool result

final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id,
         "content": '{"temp_c": 32, "cond": "sunny"}'},
    ],
    tools=tools,
)

print(final.choices[0].message.content)

Vision and Multimodal Inputs

When messages include image_url content blocks, the router restricts selection to providers declaring Vision capability. If no vision-enabled providers are available, the system returns HTTP 422 with error code no_vision_model.

The capability filtering logic in server/src/routes/proxy.ts runs before invoking the provider client to ensure compatibility.

Complete Code Examples

Basic Python Integration

Use the official OpenAI SDK by pointing the base URL to your FreeLLMAPI instance:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

cURL Requests

Test the endpoint directly without SDKs:

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "hi"}]
      }'

Debugging and Observability

Every response includes the X-Routed-Via header indicating the actual provider and model that served the request. Optional fallback-detail headers provide additional debugging information when the system retries across multiple providers.

For complete schema documentation, access the interactive Swagger UI at GET /v1/docs or download the OpenAPI specification at GET /v1/openapi.json.

Summary

• Drop-in compatibility: The /v1/chat/completions endpoint in server/src/routes/proxy.ts follows the exact OpenAI specification, supporting standard SDKs without code changes
• Intelligent routing: The auto model selection leverages server/src/lib/fallback-loop.ts to provide resilient failover across free-tier providers
• Full feature parity: Supports streaming responses, tool calling, and vision inputs through the generic provider client in server/src/providers/openai-compat.ts
• Unified security: Single Bearer token authentication maps to multiple provider keys via the dashboard's key management system

Frequently Asked Questions

How does FreeLLMAPI handle rate limits from underlying providers?

When a provider returns a rate limit or error, the proxy router automatically retries the request with the next provider in the fallback chain defined in server/src/lib/fallback-loop.ts. This happens transparently within the single request lifecycle, returning the first successful response to your client without exposing intermediate failures.

Can I use the standard OpenAI Python SDK with FreeLLMAPI?

Yes. FreeLLMAPI is designed to be drop-in compatible with OpenAI's official SDKs. Simply set the base_url to http://localhost:3001/v1 and provide your unified API key. All standard methods including chat.completions.create(), streaming iterators, and tool calling work without modifications.

What happens if I request vision capabilities but no providers support images?

The system returns an HTTP 422 error with code no_vision_model. According to the implementation in server/src/routes/proxy.ts, the router detects image_url content blocks and verifies that at least one enabled provider declares Vision capability before forwarding. If none exist, it fails fast with a clear error message.

Is there an interactive API documentation interface available?

Yes. Access the interactive Swagger UI at GET /v1/docs or download the raw OpenAPI specification at GET /v1/openapi.json. These endpoints, maintained alongside the proxy implementation, provide complete schema definitions for the /v1/chat/completions endpoint including all supported parameters for routing strategies, streaming, and tool use.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}