How to Integrate FreeLLMAPI with Existing Applications: A Complete Developer's Guide

Integrate FreeLLMAPI with existing applications by pointing any OpenAI-compatible client to http://localhost:3001/v1 and authenticating with your unified fla_ API key, enabling automatic routing across dozens of free-tier LLM providers without code changes.

FreeLLMAPI exposes a single OpenAI-compatible endpoint that aggregates free-tier LLM providers behind one unified API key. According to the tashfeenahmed/freellmapi source code, the routing logic lives in server/src/services/router.ts and handles provider selection, failover, and quota management automatically. This guide covers the complete integration process from server setup to advanced features like tool calling and vision inputs.

Server Setup and Installation

Before integrating your application, you must run the FreeLLMAPI server locally. The repository provides two quick-start methods.

Docker Deployment

docker compose up

One-Liner Installation Script

curl -fsSL https://freellmapi.co/install.sh | bash

The server starts on port 3001 by default and exposes the OpenAI-compatible API at /v1. This endpoint accepts standard chat completions, embeddings, and streaming requests exactly like the official OpenAI API.

Configuring Your Unified API Key

Once the server runs, generate your unified authentication token through the dashboard.

1. Navigate to the Keys page in the web interface (client/src/pages/ contains the React components for this UI).
2. Add your individual provider keys (Groq, Mistral, Google, etc.); the system stores them encrypted in SQLite.
3. Copy the generated unified API key (the fla_… token).

This single key replaces all individual provider credentials in your application code. The router in server/src/services/router.ts uses these stored keys to make requests on your behalf while enforcing headroom guardrails so you never exceed free tier limits.

Connecting OpenAI-Compatible Clients

FreeLLMAPI requires only two configuration changes in existing applications: updating the base_url to http://localhost:3001/v1 and switching to your unified API key. All other request parameters—including messages, tool definitions, and streaming options—pass through unchanged.

Python OpenAI SDK Integration

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",               # ← FreeLLMAPI base

    api_key="freellmapi-your-unified-key",             # ← Generated in dashboard

)

resp = client.chat.completions.create(
    model="auto",                                       # ← Let router pick

    messages=[{"role": "user", "content": "Summarise the fall of Rome"}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

The response includes an x-routed-via header indicating which provider handled the request.

cURL Integration

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "Explain quantum tunnelling"}]
      }'

Advanced Routing and Features

The integration supports sophisticated routing strategies and OpenAI-compatible features without additional client configuration.

Auto-Routing Strategies

The router supports multiple selection strategies via the model parameter:

• auto – Balanced selection based on reliability, speed, and intelligence scores
• auto:fast – Prioritizes lowest latency providers
• auto:smart – Prioritizes highest capability models
• auto:reliable – Prioritizes providers with best uptime records
• auto:balanced – Weighs speed and capability equally

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto:fast",
        "messages": [{"role": "user", "content": "Write a quick sort in Rust"}]
      }'

The routing logic in server/src/services/router.ts evaluates provider health, quota headroom, and historical performance to make these selections automatically.

Tool Calling Implementation

Tool calling works identically to the OpenAI API. The router forwards function definitions to the selected provider and handles the response:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
    },
}]

first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="required",
)
call = first.choices[0].message.tool_calls[0]

# Execute locally, then feed result back

final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Tokyo?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id, "content": '{"temp_c": 22, "cond":"clear"}'},
    ],
    tools=tools,
)
print(final.choices[0].message.content)

Vision and Multimodal Inputs

Vision requests route to providers capable of image processing:

resp = client.chat.completions.create(
    model="auto",   # router will pick a vision-capable model

    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this picture?"},
            {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64_DATA>"}}
        ],
    }],
)
print(resp.choices[0].message.content)

Embedding Requests

Embedding endpoints use the same unified key but do not fall back across model families:

emb = client.embeddings.create(
    model="auto",                           # uses the default embedding family

    input=["the quick brown fox", "jumps over the lazy dog"],
)
print(len(emb.data), "vectors of", len(emb.data[0].embedding), "dims")

CLI Setup for Popular Coding Agents

FreeLLMAPI includes setup commands for popular AI coding assistants. These commands modify your agent configuration files to use the local endpoint.

# Claude Code

npx freellmapi setup-claude --url http://localhost:3001 --api-key freellmapi-your-unified-key

# Codex CLI

npx freellmapi setup-codex --url http://localhost:3001 --api-key freellmapi-your-unified-key

# Aider

npx freellmapi setup-aider --url http://localhost:3001 --api-key freellmapi-your-unified-key

All generators support --dry-run and create timestamped backups before modifying existing configurations.

Understanding the Routing Architecture

Several key source files govern the integration behavior:

• server/src/services/router.ts – Implements model selection, penalty tracking, headroom calculations, and fallback logic when providers return rate limits or 5xx errors.
• server/src/providers/* (e.g., openai-compat.ts) – Wraps each free provider behind the unified interface, normalizing request/response formats.
• server/src/db/index.ts and server/src/scoring.ts – Store per-model scores, headroom thresholds, and custom routing weights used by the selection algorithm.
• docs/api.md – Documents concrete request/response shapes for OpenAI-compatible endpoints, streaming, and tool calling.

Because the routing logic is fully contained in the server, you do not need to change your existing code beyond the endpoint URL and API key.

Summary

• FreeLLMAPI aggregates dozens of free-tier LLM providers behind a single OpenAI-compatible endpoint at http://localhost:3001/v1.
• Installation requires only Docker (docker compose up) or a one-liner bash script.
• Authentication uses a unified fla_ API key generated in the dashboard's Keys page, replacing individual provider credentials.
• Integration works with any OpenAI-compatible client (Python SDK, cURL, JavaScript, Anthropic SDK) by changing only the base_url and api_key.
• Auto-routing supports strategies like auto:fast, auto:smart, and auto:reliable via the model parameter.
• Advanced features including tool calling, vision inputs, and embeddings pass through transparently with automatic provider selection.

Frequently Asked Questions

What is the difference between FreeLLMAPI and using individual provider APIs directly?

FreeLLMAPI eliminates the need to manage multiple API keys, track quota limits, and implement fallback logic in your application. According to the server/src/services/router.ts implementation, the service automatically routes requests to the healthiest available provider and switches to backups when rate limits occur, all while using a single unified key.

Can I use FreeLLMAPI with the official OpenAI Python library?

Yes. FreeLLMAPI is fully compatible with the official openai Python package and Node.js SDK. Simply change the base_url to http://localhost:3001/v1 and provide your unified API key. The router handles translation between the OpenAI request format and each provider's specific requirements through the adapters in server/src/providers/.

How does the auto-routing model selection work?

The router evaluates providers based on four criteria: reliability, speed, intelligence, and current quota headroom. When you specify model="auto" or variants like auto:fast, the algorithm in server/src/scoring.ts calculates weighted scores and selects the optimal provider for your specific request type.

Is my data secure when using FreeLLMAPI?

Provider API keys are encrypted at rest in the local SQLite database (server/src/db/index.ts). Requests pass through your local server instance; individual provider keys never leave your infrastructure. The dashboard UI in client/src/pages/ handles key storage using encryption before the data reaches the database layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}