How to Use FreeLLMAPI with the OpenAI Python Client: Complete Integration Guide

Set the base_url to your FreeLLMAPI endpoint (e.g., http://localhost:3001/v1) and use your unified FreeLLMAPI key as the api_key to route requests through the proxy while maintaining full compatibility with the official OpenAI SDK.

FreeLLMAPI (tashfeenahmed/freellmapi) exposes a single OpenAI-compatible endpoint at /v1 that proxies requests to free-tier language models. By configuring the OpenAI Python client to point at this endpoint, you leverage automatic model routing, rate-limit tracking, and encrypted upstream key handling without modifying your existing application logic.

Configuring the OpenAI Client

To use FreeLLMAPI with the OpenAI Python client, you only need to change two initialization parameters. Set base_url to your FreeLLMAPI server address (including the /v1 path), and provide your unified FreeLLMAPI key as the api_key. The proxy handles all upstream provider authentication internally, so you never expose your original provider keys in client-side code.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",      # FreeLLMAPI endpoint

    api_key="freellmapi-your-unified-key",    # From dashboard Keys page

)

According to the source code in server/src/routes/proxy.ts, all standard OpenAI routes—including /chat/completions, /models, and embeddings—are mounted under this /v1 base path.

Sending Chat Completion Requests

Once configured, you invoke the standard chat.completions.create method exactly as you would with the official OpenAI API.

Synchronous Non-Streaming Requests

Use model="auto" to let the router select the best available free model, or specify a model ID directly to pin a specific provider.

response = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarize the fall of Rome in one sentence."}],
)

print(response.choices[0].message.content)
print("Routed via:", response.headers.get("x-routed-via"))  # Provider identification

The x-routed-via header reveals which upstream provider answered the request, useful for debugging provider-specific behavior.

Streaming Real-Time Responses

Enable streaming by setting stream=True. FreeLLMAPI passes through the provider’s Server-Sent Events (SSE) stream unchanged via the handler in server/src/providers/openai-compat.ts.

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Working with Advanced OpenAI Features

FreeLLMAPI translates OpenAI wire formats to appropriate backend providers, supporting tools, vision, and multimodal workflows.

Function Calling and Tool Use

Define your tool schema and pass it to the tools parameter. The proxy forwards tools and tool_choice parameters to the selected backend and returns tool_calls in the standard OpenAI format.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"]
        },
    },
}]

# Initial request asking for tool call

first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required",
)

call = first.choices[0].message.tool_calls[0]

# Follow-up with tool result to get final answer

final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id,
         "content": '{"temp_c": 32, "cond": "sunny"}'},
    ],
    tools=tools,
)

print(final.choices[0].message.content)

Vision and Multimodal Inputs

Send image URLs or base64-encoded data using the standard OpenAI vision schema. The router in server/src/services/router.ts automatically routes to vision-capable models when it detects image content.

response = client.chat.completions.create(
    model="auto",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What does this picture show?"},
            {"type": "image_url",
             "image_url": {"url": "data:image/png;base64,<BASE64_DATA>"}},
        ],
    }],
)

If no vision-capable model is enabled in your configuration, FreeLLMAPI returns a clear 422 error rather than silently failing.

Understanding the Internal Routing Flow

Under the hood, FreeLLMAPI processes your OpenAI-compatible request through three key components:

1. server/src/routes/proxy.ts – Exposes the /v1/* endpoints and validates your bearer token.
2. server/src/services/router.ts – Selects the optimal model and upstream key based on current rate limits and availability.
3. server/src/providers/openai-compat.ts – Forwards the request to the chosen provider while preserving the OpenAI request shape, then reshapes the response back to strict OpenAI format.

This architecture ensures that from the Python client’s perspective, you are communicating with a standard OpenAI API endpoint.

Summary

• Base URL: Point the OpenAI client to http://localhost:3001/v1 (or your deployed URL) to access all standard endpoints.
• Authentication: Use your unified freellmapi-... key; upstream provider keys remain encrypted server-side.
• Model Selection: Set model="auto" to leverage intelligent routing, or specify exact model IDs for deterministic behavior.
• Feature Parity: Streaming, function calling, vision inputs, and tool workflows function identically to the native OpenAI API.
• Transparency: Inspect the x-routed-via response header to identify which upstream provider handled your request.

Frequently Asked Questions

Do I need to rewrite my existing OpenAI code to use FreeLLMAPI?

No. You only need to modify the client initialization to include the base_url and your FreeLLMAPI key. All other method calls—including chat.completions.create, streaming, and tool usage—remain identical to the standard OpenAI SDK implementation.

What happens if the selected free model hits its rate limit?

The router.ts service automatically enforces per-key quotas and implements failover logic. If your initially selected provider is rate-limited, the proxy attempts to route to an alternative available model before returning an error, ensuring higher availability for your requests.

Can I use FreeLLMAPI with other OpenAI-compatible SDKs besides Python?

Yes. Because FreeLLMAPI exposes a standard OpenAI REST API at /v1, any HTTP client or SDK that supports the OpenAI specification—including JavaScript, Go, or curl—can connect by setting the same base_url and authorization headers.

How does FreeLLMAPI secure my upstream API keys?

Your original provider keys (e.g., actual API keys for OpenAI, Anthropic, or Google) are encrypted and stored server-side. You only expose the unified FreeLLMAPI key in client code. The proxy decrypts and injects the appropriate upstream credentials internally in openai-compat.ts, ensuring your raw provider keys never travel over the network to client devices.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}