# How to Use FreeLLMAPI with the OpenAI Python Client: Complete Integration Guide

> Easily integrate FreeLLMAPI with the OpenAI Python client. Set your FreeLLMAPI endpoint and key to access models seamlessly using the official SDK.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-27

---

**Set the `base_url` to your FreeLLMAPI endpoint (e.g., `http://localhost:3001/v1`) and use your unified FreeLLMAPI key as the `api_key` to route requests through the proxy while maintaining full compatibility with the official OpenAI SDK.**

FreeLLMAPI (tashfeenahmed/freellmapi) exposes a single OpenAI-compatible endpoint at `/v1` that proxies requests to free-tier language models. By configuring the OpenAI Python client to point at this endpoint, you leverage automatic model routing, rate-limit tracking, and encrypted upstream key handling without modifying your existing application logic.

## Configuring the OpenAI Client

To use FreeLLMAPI with the OpenAI Python client, you only need to change two initialization parameters. Set `base_url` to your FreeLLMAPI server address (including the `/v1` path), and provide your **unified** FreeLLMAPI key as the `api_key`. The proxy handles all upstream provider authentication internally, so you never expose your original provider keys in client-side code.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",      # FreeLLMAPI endpoint

    api_key="freellmapi-your-unified-key",    # From dashboard Keys page

)

```

According to the source code in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), all standard OpenAI routes—including `/chat/completions`, `/models`, and embeddings—are mounted under this `/v1` base path.

## Sending Chat Completion Requests

Once configured, you invoke the standard `chat.completions.create` method exactly as you would with the official OpenAI API.

### Synchronous Non-Streaming Requests

Use `model="auto"` to let the **router** select the best available free model, or specify a model ID directly to pin a specific provider.

```python
response = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize the fall of Rome in one sentence."}],
)

print(response.choices[0].message.content)
print("Routed via:", response.headers.get("x-routed-via"))  # Provider identification

```

The `x-routed-via` header reveals which upstream provider answered the request, useful for debugging provider-specific behavior.

### Streaming Real-Time Responses

Enable streaming by setting `stream=True`. FreeLLMAPI passes through the provider’s Server-Sent Events (SSE) stream unchanged via the handler in [`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts).

```python
stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

```

## Working with Advanced OpenAI Features

FreeLLMAPI translates OpenAI wire formats to appropriate backend providers, supporting tools, vision, and multimodal workflows.

### Function Calling and Tool Use

Define your tool schema and pass it to the `tools` parameter. The proxy forwards `tools` and `tool_choice` parameters to the selected backend and returns `tool_calls` in the standard OpenAI format.

```python
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"]
        },
    },
}]

# Initial request asking for tool call

first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required",
)

call = first.choices[0].message.tool_calls[0]

# Follow-up with tool result to get final answer

final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id,
         "content": '{"temp_c": 32, "cond": "sunny"}'},
    ],
    tools=tools,
)

print(final.choices[0].message.content)

```

### Vision and Multimodal Inputs

Send image URLs or base64-encoded data using the standard OpenAI vision schema. The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) automatically routes to vision-capable models when it detects image content.

```python
response = client.chat.completions.create(
    model="auto",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What does this picture show?"},
            {"type": "image_url",
             "image_url": {"url": "data:image/png;base64,<BASE64_DATA>"}},
        ],
    }],
)

```

If no vision-capable model is enabled in your configuration, FreeLLMAPI returns a clear `422` error rather than silently failing.

## Understanding the Internal Routing Flow

Under the hood, FreeLLMAPI processes your OpenAI-compatible request through three key components:

1. **[`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts)** – Exposes the `/v1/*` endpoints and validates your bearer token.
2. **[`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)** – Selects the optimal model and upstream key based on current rate limits and availability.
3. **[`server/src/providers/openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/openai-compat.ts)** – Forwards the request to the chosen provider while preserving the OpenAI request shape, then reshapes the response back to strict OpenAI format.

This architecture ensures that from the Python client’s perspective, you are communicating with a standard OpenAI API endpoint.

## Summary

- **Base URL**: Point the OpenAI client to `http://localhost:3001/v1` (or your deployed URL) to access all standard endpoints.
- **Authentication**: Use your unified `freellmapi-...` key; upstream provider keys remain encrypted server-side.
- **Model Selection**: Set `model="auto"` to leverage intelligent routing, or specify exact model IDs for deterministic behavior.
- **Feature Parity**: Streaming, function calling, vision inputs, and tool workflows function identically to the native OpenAI API.
- **Transparency**: Inspect the `x-routed-via` response header to identify which upstream provider handled your request.

## Frequently Asked Questions

### Do I need to rewrite my existing OpenAI code to use FreeLLMAPI?

No. You only need to modify the client initialization to include the `base_url` and your FreeLLMAPI key. All other method calls—including `chat.completions.create`, streaming, and tool usage—remain identical to the standard OpenAI SDK implementation.

### What happens if the selected free model hits its rate limit?

The [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) service automatically enforces per-key quotas and implements failover logic. If your initially selected provider is rate-limited, the proxy attempts to route to an alternative available model before returning an error, ensuring higher availability for your requests.

### Can I use FreeLLMAPI with other OpenAI-compatible SDKs besides Python?

Yes. Because FreeLLMAPI exposes a standard OpenAI REST API at `/v1`, any HTTP client or SDK that supports the OpenAI specification—including JavaScript, Go, or curl—can connect by setting the same `base_url` and authorization headers.

### How does FreeLLMAPI secure my upstream API keys?

Your original provider keys (e.g., actual API keys for OpenAI, Anthropic, or Google) are encrypted and stored server-side. You only expose the unified FreeLLMAPI key in client code. The proxy decrypts and injects the appropriate upstream credentials internally in [`openai-compat.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/openai-compat.ts), ensuring your raw provider keys never travel over the network to client devices.