How to Set Up FreeLLMAPI with Docker: Complete Self-Hosting Guide

Running FreeLLMAPI in Docker spins up a single container that hosts an OpenAI-compatible API proxy, encrypted key storage, intelligent request routing, and a React dashboard—accessible on localhost in under five minutes.

FreeLLMAPI is a self-hosted aggregation layer from the tashfeenahmed/freellmapi repository that unifies dozens of free-tier LLM providers behind one endpoint. When you deploy it via Docker Compose, you get a persistent, production-ready instance featuring automatic key encryption, rate-limit management, and a web-based admin panel for managing your provider fallback chains.

Architecture Overview

The Docker image bundles several components into a single multi-arch (linux/amd64, linux/arm64) container defined in the Dockerfile:

• Express Server (server/src/services/router.ts) – Handles /v1/* API routes, decrypts stored keys using AES-256-GCM, and forwards requests to selected providers.
• Intelligent Router (server/src/services/router.ts) – Selects the best-available model based on health checks, rate limits, and your configured fallback chain.
• Rate-Limit Ledger (server/src/services/ratelimit.ts) – Tracks per-key RPM/RPD/TPM/TPD counters in SQLite with automatic cooldown handling for 429/5xx responses.
• Provider Adapters (server/src/providers/*.ts) – Modular implementations for each LLM service (Google, Groq, OpenRouter, etc.).
• Dashboard UI (client/) – React + Vite interface served from the same Express instance for key management and analytics.
• Encrypted Key Store (server/data/) – SQLite database protected by your ENCRYPTION_KEY environment variable.

Step-by-Step Docker Installation

1. Prepare Your Deployment Directory

Create a dedicated folder for your configuration and data volume:

mkdir ~/freellmapi && cd ~/freellmapi

2. Generate an Encryption Key

The server requires a 32-byte hex key to encrypt provider API keys at rest. Generate it and create your environment file:

ENCRYPTION_KEY=$(openssl rand -hex 32)
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env

The [docker-compose.yml](https://github.com/tashfeenahmed/freellmapi/blob/main/docker-compose.yml) automatically reads this .env file on startup.

3. Launch the Container

Pull the pre-built image and start the service:

docker compose up -d

This exposes the unified API and dashboard on http://localhost:3001. The container mounts a named volume (freellmapi-data) to persist your encrypted SQLite database across restarts.

4. Configure Network Access (Optional)

By default, the container binds to 127.0.0.1 only. To expose FreeLLMAPI to other devices on your LAN:

HOST_BIND=0.0.0.0 docker compose up -d

5. Initialize the Dashboard

Navigate to http://localhost:3001 and complete the setup:

1. Navigate to the Keys page and add your provider API keys.
2. Arrange the Fallback Chain to prioritize preferred models.
3. Copy the unified bearer token displayed at the top—this authenticates all client requests.

Connecting Your Applications

Once running, point any OpenAI-compatible client at http://<host>:3001/v1 using the unified token from your dashboard.

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise the fall of Rome."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

cURL Requests

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "Hi there!"}]
      }'

Streaming Responses

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Key Configuration Files

Understanding these source files helps with customization and troubleshooting:

• Dockerfile – Builds the all-in-one production image containing both the Express server and compiled React dashboard.
• docker-compose.yml – Orchestrates the container, mounts the persistent data volume, and injects environment variables from .env.
• server/src/services/router.ts – Core routing logic implementing model selection, health checks, and fallback handling.
• server/src/services/ratelimit.ts – Implements SQLite-backed rate limiting with per-key cooldown management.
• server/src/providers/ – Directory containing provider-specific adapters for each supported LLM service.
• client/ – Source code for the React+Vite admin panel used for key management and analytics.

Summary

• FreeLLMAPI packages a proxy server, encrypted storage, and admin UI into a single Docker container from tashfeenahmed/freellmapi.
• Security relies on a 32-byte ENCRYPTION_KEY set via .env to protect provider keys using AES-256-GCM.
• Persistence uses a named Docker volume (freellmapi-data) mounted to server/data/ for the SQLite database.
• Routing logic in server/src/services/router.ts automatically handles health checks and rate-limit cooldowns.
• Compatibility is OpenAI-compliant—use any standard client library by changing the base_url and api_key.

Frequently Asked Questions

How do I back up my FreeLLMAPI data?

The encrypted SQLite database resides in a Docker named volume (freellmapi-data). Back up this volume using docker run --rm -v freellmapi-data:/data -v $(pwd):/backup alpine tar czf /backup/freellmapi-backup.tar.gz /data or copy the server/data/ directory if running with bind mounts.

Can I run FreeLLMAPI without the dashboard?

While the dashboard is compiled into the Docker image and served from the Express server in server/src/services/router.ts, you can disable access by blocking the root path at your reverse proxy. The API endpoints under /v1/* remain fully functional without UI interaction.

Why does my container exit with an encryption error?

The server refuses to start if the ENCRYPTION_KEY environment variable is missing or invalid. Ensure your .env file contains a 64-character hex string (32 bytes) generated via openssl rand -hex 32, and that docker-compose.yml is in the same directory so Docker Compose reads the file automatically.

How are rate limits tracked across restarts?

Per-provider RPM/RPD/TPM/TPD counters are stored in SQLite at server/data/ and mounted via the freellmapi-data volume. This persists rate-limit state and cooldown timers across container restarts, as implemented in server/src/services/ratelimit.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}