How to Integrate FreeLLM API into Your Project: A Complete Self-Hosted Setup Guide

FreeLLM API is a self-hosted proxy that consolidates free-tier access to dozens of LLM providers behind a single OpenAI-compatible endpoint, allowing you to route requests through http://localhost:3001/v1 using a unified API key.

FreeLLM API (tashfeenahmed/freellmapi) eliminates the complexity of managing multiple API keys across different providers by exposing a standardized interface that any OpenAI-compatible client can consume. When you integrate FreeLLM API into your project, you gain automatic load balancing across 161+ free models, intelligent fallback handling, and unified authentication without rewriting existing request logic.

Deployment Options

The server listens on http://localhost:3001/v1 by default and stores provider credentials in an encrypted SQLite database. You can deploy via Docker or run directly from source.

Docker Deployment

Clone the repository and use the provided Compose configuration:

git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi/docker
docker compose up -d

Refer to [docker/README.md](https://github.com/tashfeenahmed/freellmapi/blob/main/docker/README.md) for environment-specific configurations and volume mappings.

Source Installation

Install dependencies and start the development server:

npm install
npm run dev

The application reads configuration from .env, including the ENCRYPTION_KEY used to secure provider credentials in the SQLite database and the server port (defaulting to 3001).

Configuring Provider Keys

Once the server is running, open the dashboard at http://localhost:3001 and navigate to the Keys page. Paste your free-tier API keys from providers such as Google, Groq, Mistral, and OpenRouter.

The dashboard encrypts these credentials using the ENCRYPTION_KEY defined in your environment variables before persisting them to the database, ensuring provider secrets remain secure at rest.

Obtaining the Unified API Key

The dashboard displays a freellmapi-… token at the top of the Keys page. This single token authenticates all downstream requests to the proxy, replacing the need to manage multiple provider-specific keys in your application code.

Client Integration

To integrate FreeLLM API, point any OpenAI-compatible SDK to http://localhost:3001/v1 and use the unified token as the api_key. The router automatically selects the best available model, handles rate-limit errors, and maintains the same request/response schema as the official OpenAI API.

Python OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",   # ⭐ point to FreeLLMAPI

    api_key="freellmapi-xxxxxxxxxxxxxxxxx" # unified token from dashboard

)

resp = client.chat.completions.create(
    model="auto",  # let the router choose the best free model

    messages=[{"role": "user",
               "content": "Summarise the fall of Rome in one sentence."}]
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

curl Requests

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-xxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "hi"}]
      }'

VS Code Continue Extension

models:
  - name: FreeLLMAPI Autocomplete
    provider: openai
    model: auto
    apiBase: http://localhost:3001/v1
    apiKey: freellmapi-xxxxxxxxxxxxxxxxx
    useLegacyCompletionsEndpoint: true
    roles:
      - autocomplete

Advanced Features

FreeLLM API supports the full OpenAI API specification including streaming, tool calling, and multimodal endpoints.

Tool Calling and Function Calls

Function calls pass through unchanged to the underlying providers. The proxy handles the request/response cycle transparently:

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"]
        }
    }
}]

first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required"
)

call = first.choices[0].message.tool_calls[0]

# Simulate tool execution, then feed result back

final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id,
         "content": '{"temp_c": 32, "cond": "sunny"}'}
    ],
    tools=tools,
)
print(final.choices[0].message.content)

Media Endpoints and Embeddings

The proxy routes vision and audio requests to providers supporting media generation through endpoints like /v1/images/generations and /v1/audio/speech. For embeddings, the system routes by model family in [server/src/services/router.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) to prevent mixing incompatible vector spaces.

Streaming responses (stream=True) work unchanged through the proxy layer.

Architecture Overview

Understanding the core components helps when debugging or extending the integration.

Request Routing: The [server/src/services/router.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) file contains the selection logic that evaluates provider health, quota availability, and priority to determine the optimal model for each request.

Rate Limiting: Per-provider rate tracking (RPM, RPD, TPM, TPD) and cooldown handling occur in [server/src/services/ratelimit.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), preventing quota exhaustion across the pooled free tiers.

Provider Adapters: Low-level SDK integrations for specific services (Groq, Google, OpenRouter, etc.) reside in the server/src/providers/ directory, with individual implementations like [server/src/providers/groq.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/providers/groq.ts) handling provider-specific request formatting.

Health Monitoring: The [server/src/services/health.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) service performs periodic validation of configured provider keys to ensure availability before routing.

Environment Configuration: Server settings including port, encryption keys, and database paths are managed in [server/src/env.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/env.ts).

Dashboard UI: The React-based frontend code and usage documentation reside in [client/README.md](https://github.com/tashfeenahmed/freellmapi/blob/main/client/README.md).

Summary

• FreeLLM API provides an OpenAI-compatible proxy that aggregates free-tier LLM access behind a single endpoint at http://localhost:3001/v1.
• Deployment supports both Docker (docker compose up -d) and Node.js source (npm install && npm run dev), with credentials encrypted in SQLite using an ENCRYPTION_KEY.
• Authentication uses a unified freellmapi-… token from the dashboard, eliminating the need to embed multiple provider keys in your application.
• Integration requires only changing the base_url and api_key in any OpenAI SDK client; the model="auto" parameter enables intelligent routing across 161+ models.
• Advanced capabilities include streaming, tool calling, vision/audio generation, and embeddings routing by model family, all handled transparently by the proxy architecture.

Frequently Asked Questions

Do I need to modify my existing OpenAI SDK code to use FreeLLM API?

No. FreeLLM API maintains full compatibility with the OpenAI API specification. You only need to change the base_url (or apiBase) to http://localhost:3001/v1 and replace the api_key with your unified freellmapi-… token. All existing request patterns, including streaming and tool calling, function identically without code changes.

How does FreeLLM API handle rate limits across multiple providers?

The system tracks rate limits (RPM, RPD, TPM, TPD) individually for each provider in [server/src/services/ratelimit.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts). When a provider approaches its quota, the router in [server/src/services/router.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) automatically fails over to the next available model, ensuring continuous service without manual intervention.

Can I use FreeLLM API with applications other than Python, such as Node.js or VS Code extensions?

Yes. Any client supporting OpenAI-compatible endpoints works with FreeLLM API, including Node.js applications, curl scripts, VS Code Continue extension, and Claude Desktop. The unified API key and standard REST interface make it compatible with any HTTP client or SDK that can specify a custom base URL.

Is provider credential storage secure?

Yes. Provider API keys are encrypted using AES-256 via the ENCRYPTION_KEY environment variable before being stored in the local SQLite database. The encryption occurs in the dashboard interface, and keys are only decrypted in-memory when making requests to specific providers, as configured in [server/src/env.ts](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/env.ts).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}