How to Set Up FreeLLMAPI Locally Using Docker: A Complete Guide

To set up FreeLLMAPI locally using Docker, generate an AES-256 encryption key, create a .env file with your configuration, and run docker compose up -d to launch the OpenAI-compatible proxy server on port 3001.

FreeLLMAPI is a self-hosted proxy that aggregates free tiers from dozens of LLM providers into a single OpenAI-compatible endpoint. Setting it up locally using Docker provides a complete, containerized environment that includes the Express API server, React dashboard, and encrypted SQLite database. This guide walks through the exact steps to deploy the service using the official docker-compose.yml configuration from the tashfeenahmed/freellmapi repository.

What FreeLLMAPI Deploys

Running the Docker container initializes several interconnected components automatically:

• Express Server – Handles all /v1/* API routes, decrypts stored provider keys, and forwards requests to upstream services (implemented in server/src/services/router.ts)
• Intelligent Router – Selects the best-available model based on health checks, rate limits, and your configured fallback chain (also in server/src/services/router.ts)
• Rate-Limit Ledger – Tracks per-key RPM/RPD/TPM/TPD counters in SQLite with automatic cooldown handling for 429/5xx responses (server/src/services/ratelimit.ts)
• Provider Adapters – Modular implementations for each LLM provider (server/src/providers/*.ts)
• Dashboard UI – React + Vite admin panel for key management and analytics (client/)
• Encrypted Key Store – AES-256-GCM encrypted SQLite database mounted via Docker volume (server/data/)

Prerequisites

• Docker Engine 20.10+ and Docker Compose 2.0+ installed
• openssl command-line tool for generating encryption keys
• Approximately 500MB free disk space for the multi-arch image (linux/amd64 or linux/arm64)

Step-by-Step Docker Installation

1. Prepare the Deployment Directory

Create a dedicated directory for your deployment to isolate the environment file and Docker volumes:

mkdir ~/freellmapi && cd ~/freellmapi

2. Generate the Encryption Key

The server requires a 32-byte hex encryption key to secure provider API keys at rest. Generate this and create the .env file:

ENCRYPTION_KEY=$(openssl rand -hex 32)
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env

The docker-compose.yml automatically reads variables from this .env file.

3. Start the Container

Pull the pre-built image and start the services in detached mode:

docker compose up -d

This command:

• Downloads the multi-arch image containing both server and compiled dashboard
• Creates a named volume freellmapi-data for persistent SQLite storage
• Exposes the API and dashboard on http://localhost:3001

4. Configure Network Access (Optional)

By default, the container binds only to 127.0.0.1. To expose the service on your LAN for other devices, prepend the HOST_BIND variable:

HOST_BIND=0.0.0.0 docker compose up -d

5. Initialize the Dashboard

Navigate to http://localhost:3001 (or your LAN address) and complete the setup:

1. Add Provider Keys – Visit the Keys page to input your free-tier API keys for supported providers
2. Configure Fallback Chain – Arrange provider priority on the Fallback Chain page
3. Copy Unified Token – Retrieve the bearer token displayed at the top of the Keys page for API authentication

Connecting to the API

Once running, FreeLLMAPI presents an OpenAI-compatible endpoint at /v1.

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",  # From dashboard

)

resp = client.chat.completions.create(
    model="auto",  # Router selects best available

    messages=[{"role": "user", "content": "Explain quantum computing."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

curl

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "Hello!"}]
      }'

Streaming Responses

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream a haiku about Docker."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Key Configuration Files

Understanding these files helps customize your deployment:

• docker-compose.yml – Orchestrates the container, mounts the freellmapi-data volume, and injects environment variables from .env
• Dockerfile – Builds the production image bundling the Express server and compiled React dashboard
• server/src/services/router.ts – Contains the core routing logic for model selection and fallback handling
• server/src/services/ratelimit.ts – Implements rate-limit tracking and cooldown management
• server/src/providers/ – Directory containing provider-specific adapter implementations
• docker/README.md – Additional troubleshooting and advanced configuration options

Summary

• FreeLLMAPI aggregates multiple free LLM providers behind a single OpenAI-compatible endpoint using Docker
• Encryption is mandatory – Generate a 32-byte hex key via openssl rand -hex 32 before starting the container
• Single command deployment – docker compose up -d starts the API server and dashboard on port 3001
• Persistent storage – The SQLite database mounts to freellmapi-data volume for encrypted key storage
• LAN access – Bind to 0.0.0.0 using the HOST_BIND environment variable to expose the service beyond localhost

Frequently Asked Questions

How do I back up the FreeLLMAPI database?

The SQLite database persists in a Docker named volume (freellmapi-data). You can back it up by copying the volume contents or using docker cp to extract the server/data/ directory from the running container. The database contains AES-256-GCM encrypted keys, so ensure your backup includes the ENCRYPTION_KEY from your .env file.

What port does FreeLLMAPI use by default?

The default port is 3001, configured via the PORT environment variable in your .env file. Both the REST API and the React dashboard are served from this single port. The Express server (server/src/services/router.ts) handles API routes while static assets are served from the compiled client/ build.

How do I update the FreeLLMAPI Docker container?

Pull the latest image and restart the services while preserving your data volume:

docker compose pull
docker compose up -d

Your encrypted provider keys and rate-limit data remain intact in the freellmapi-data volume.

Where are the provider API keys stored?

Provider keys are stored in an AES-256-GCM encrypted SQLite database at server/data/ inside the container, backed by the freellmapi-data Docker volume. The encryption key is never stored in the database; it must be provided via the ENCRYPTION_KEY environment variable at runtime, as implemented in the key storage service.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}