# How to Use the FreeLLMAPI /v1/chat/completions Endpoint: A Complete Guide

> Learn to use the FreeLLMAPI v1 chat completions endpoint to route requests through free-tier providers like Groq, Cohere, or Google Gemini with automatic model selection and a unified Bearer token.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-26

---

**Use the OpenAI-compatible endpoint at `http://localhost:3001/v1/chat/completions` with a unified Bearer token to route chat requests through free-tier providers like Groq, Cohere, or Google Gemini using automatic model selection.**

FreeLLMAPI acts as a unified gateway that transforms multiple free LLM providers into a single OpenAI-compatible interface. This article explains how to interact with the `/v1/chat/completions` endpoint based on the implementation in the `tashfeenahmed/freellmapi` repository, covering authentication, intelligent routing, streaming, and advanced features.

## Endpoint Basics and Authentication

The chat completions endpoint follows the standard OpenAI API specification, ensuring compatibility with existing SDKs and third-party tools.

### Base URL and Unified Key Authentication

All requests target:

```

http://localhost:3001/v1/chat/completions

```

The proxy implementation in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) validates requests using **unified key authentication**. Send your key in the `Authorization` header:

```bash
Authorization: Bearer freellmapi-your-unified-key

```

The system maps this unified key to specific provider-level credentials configured through the dashboard's *Keys* page. This abstraction allows you to rotate underlying provider keys without changing client code.

## Request Structure and Routing Logic

The request body follows the OpenAI Chat Completions schema, with FreeLLMAPI-specific routing extensions.

### The Auto-Routing System

When you specify `"model": "auto"`, the router activates the fallback chain algorithm implemented in [`server/src/lib/fallback-loop.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/fallback-loop.ts). The system iterates through enabled free providers (Groq, Cohere, Gemini), attempting the request with each until one succeeds. This ensures **resilient failover** if a provider returns errors or rate limits.

### Model Selection Strategies

Beyond simple `auto`, you can use **prefixed profiles** to control routing behavior:

- **`auto:fast`** - Prioritizes low-latency providers
- **`auto:smart`** - Selects higher-quality models  
- **`auto:balanced`** - Default behavior balancing speed and quality

According to [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), the router inspects these prefixes and selects the appropriate provider chain based on your configuration.

## Advanced Capabilities

FreeLLMAPI supports the full OpenAI feature set including streaming, tool use, and vision inputs through the generic client in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts).

### Streaming Responses

Enable streaming by setting `stream: true`. The proxy forwards Server-Sent Events (SSE) chunks exactly as OpenAI does, preserving `delta` objects and the final `finish_reason`:

```python
stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream me a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

```

### Tool Calling and Function Execution

The endpoint supports OpenAI-style function calling. The proxy passes `tools` and `tool_choice` fields unchanged to the underlying provider, then re-wraps the response into standard OpenAI format.

Implement a two-turn tool execution:

```python
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

# Request 1: Get the tool call

first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required",
)

call = first.choices[0].message.tool_calls[0]

# Request 2: Submit the tool result

final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id,
         "content": '{"temp_c": 32, "cond": "sunny"}'},
    ],
    tools=tools,
)

print(final.choices[0].message.content)

```

### Vision and Multimodal Inputs

When messages include `image_url` content blocks, the router restricts selection to providers declaring **Vision** capability. If no vision-enabled providers are available, the system returns HTTP 422 with error code `no_vision_model`.

The capability filtering logic in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) runs before invoking the provider client to ensure compatibility.

## Complete Code Examples

### Basic Python Integration

Use the official OpenAI SDK by pointing the base URL to your FreeLLMAPI instance:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

```

### cURL Requests

Test the endpoint directly without SDKs:

```bash
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "hi"}]
      }'

```

## Debugging and Observability

Every response includes the **`X-Routed-Via`** header indicating the actual provider and model that served the request. Optional fallback-detail headers provide additional debugging information when the system retries across multiple providers.

For complete schema documentation, access the interactive Swagger UI at `GET /v1/docs` or download the OpenAPI specification at `GET /v1/openapi.json`.

## Summary

- **Drop-in compatibility**: The `/v1/chat/completions` endpoint in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) follows the exact OpenAI specification, supporting standard SDKs without code changes
- **Intelligent routing**: The `auto` model selection leverages [`server/src/lib/fallback-loop.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/fallback-loop.ts) to provide resilient failover across free-tier providers
- **Full feature parity**: Supports streaming responses, tool calling, and vision inputs through the generic provider client in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts)
- **Unified security**: Single Bearer token authentication maps to multiple provider keys via the dashboard's key management system

## Frequently Asked Questions

### How does FreeLLMAPI handle rate limits from underlying providers?

When a provider returns a rate limit or error, the proxy router automatically retries the request with the next provider in the fallback chain defined in [`server/src/lib/fallback-loop.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/fallback-loop.ts). This happens transparently within the single request lifecycle, returning the first successful response to your client without exposing intermediate failures.

### Can I use the standard OpenAI Python SDK with FreeLLMAPI?

Yes. FreeLLMAPI is designed to be **drop-in compatible** with OpenAI's official SDKs. Simply set the `base_url` to `http://localhost:3001/v1` and provide your unified API key. All standard methods including `chat.completions.create()`, streaming iterators, and tool calling work without modifications.

### What happens if I request vision capabilities but no providers support images?

The system returns an HTTP 422 error with code `no_vision_model`. According to the implementation in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), the router detects `image_url` content blocks and verifies that at least one enabled provider declares Vision capability before forwarding. If none exist, it fails fast with a clear error message.

### Is there an interactive API documentation interface available?

Yes. Access the interactive Swagger UI at `GET /v1/docs` or download the raw OpenAPI specification at `GET /v1/openapi.json`. These endpoints, maintained alongside the proxy implementation, provide complete schema definitions for the `/v1/chat/completions` endpoint including all supported parameters for routing strategies, streaming, and tool use.