How to Run FreeLLMAPI in a Production Environment: Complete Deployment Guide

Run FreeLLMAPI in a production environment by deploying the official Docker container with a persistent volume for SQLite, setting a 32‑hex ENCRYPTION_KEY for credential security, and exposing the service on your internal network via HOST_BIND=0.0.0.0.

FreeLLMAPI is a self‑hosted, OpenAI‑compatible proxy developed by tashfeenahmed/freellmapi that aggregates dozens of free‑tier LLM providers behind a single /v1 endpoint. When you run FreeLLMAPI in a production environment, you create a resilient gateway that automatically routes requests, handles rate limits, and encrypts provider keys at rest. This guide walks through the architecture, deployment steps, and configuration required for a secure, production‑ready installation.

Architecture Components for Production

Understanding the core architecture helps you configure monitoring and troubleshoot issues when running FreeLLMAPI in production.

Express Proxy and Request Routing

The entry point server/src/index.ts initializes an Express server that mounts the OpenAI‑compatible /v1 routes. Incoming requests are forwarded to server/src/services/router.ts, which implements the core routing logic including model selection, sticky sessions, and automatic fallback chains. The router selects the best available model and invokes the appropriate provider adapter.

Encrypted Provider Key Storage

Provider API credentials are never stored in plain text. The system uses AES‑256‑GCM encryption implemented in server/src/db/model-pricing.ts and related database modules. The encryption key is derived from the ENCRYPTION_KEY environment variable, which must be a 64‑character hex string (32 bytes) generated via openssl rand -hex 32. Without this key, the service cannot decrypt stored credentials, making it the most critical secret for your production deployment.

Rate Limiting and Persistence

Per‑key RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day) counters are maintained in memory and persisted to SQLite. The server/src/services/ratelimit.ts module enforces these caps and triggers cooldown periods when providers return HTTP 429 or 5xx errors. By default, the SQLite database resides at /app/server/data/freellmapi.db inside the container.

Health Monitoring and Fallback

A background health‑check service defined in server/src/services/health.ts periodically probes each configured provider key to update status flags (healthy, rate_limited, invalid, error). When a request fails or hits a limit, the router automatically retries the next model in the fallback chain without client intervention.

Production Deployment Steps

Follow these steps to deploy FreeLLMAPI securely in a production environment.

1. Pull the Official Docker Image

The repository provides a multi‑arch image that bundles the compiled Node.js server and the React/Vite dashboard.

docker pull ghcr.io/tashfeenahmed/freellmapi:latest

2. Generate an Encryption Key

Generate a cryptographically secure key and store it in your .env file:

echo "ENCRYPTION_KEY=$(openssl rand -hex 32)" >> .env

3. Configure Required Environment Variables

Create a .env file based on the repository’s .env.example. At minimum, specify:

• ENCRYPTION_KEY – The 64‑character hex string from step 2
• PORT – The internal port (default 3001)
• HOST_BIND – Set to 0.0.0.0 to listen on all interfaces
• FREEAPI_DB_PATH – /app/server/data/freellmapi.db
• NODE_ENV – production (disables development fallbacks)

Optional variables include REQUEST_ANALYTICS_RETENTION_DAYS (default 90) and FREEAPI_CONFIG_PATH for declarative configuration.

4. Launch with Docker Compose

Use the official docker-compose.yml to orchestrate the container with a persistent named volume:

version: "3.8"
services:
  freellmapi:
    image: ghcr.io/tashfeenahmed/freellmapi:latest
    restart: unless-stopped
    ports:
      - "3001:3001"
    env_file: .env
    volumes:
      - freellmapi-data:/app/server/data
volumes:
  freellmapi-data:

Deploy with:

docker compose up -d

5. Add Provider Keys via the Dashboard

Once running, navigate to http://<host>:3001 to access the dashboard. Navigate to the Keys page, paste each provider’s API key (e.g., Google, Groq), and save. The UI encrypts these credentials using your ENCRYPTION_KEY before writing them to SQLite.

6. Retrieve the Unified API Key

The dashboard header displays a unified token formatted as freellmapi‑…. Use this token as the api_key in any OpenAI‑compatible client. This is the only credential your applications need; individual provider keys remain encrypted and hidden.

Production Configuration Examples

Running the Container Directly

For environments where Docker Compose is unavailable, run the container manually:

docker run -d \
  --name freellmapi \
  --restart unless-stopped \
  -p 3001:3001 \
  -e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
  -e PORT=3001 \
  -e HOST_BIND=0.0.0.0 \
  -e NODE_ENV=production \
  -v freellmapi-data:/app/server/data \
  ghcr.io/tashfeenahmed/freellmapi:latest

Python Client Configuration

Connect your applications to the production endpoint using the unified key:

from openai import OpenAI

client = OpenAI(
    base_url="http://my-prod-host:3001/v1",
    api_key="freellmapi-your-unified-key",
)

response = client.chat.completions.create(
    model="auto",  # Router selects the best available free model

    messages=[{"role": "user", "content": "Explain quantum computing"}],
)

print(response.choices[0].message.content)
print("Routed via:", response.headers.get("x-routed-via"))

Testing with curl

Verify the deployment from the command line:

curl http://my-prod-host:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"What is the capital of France?"}]}'

Scaling and Security Considerations

Horizontal Scaling

The FreeLLMAPI process is lightweight (~40 MB RSS) and stateless except for the SQLite database. To scale horizontally:

1. Mount the same named volume (freellmapi-data) to all replicas, or migrate to an external database.
2. Ensure all replicas share the identical ENCRYPTION_KEY.
3. Use a load balancer to distribute requests across replicas.

For Kubernetes deployments, specify the volume claim for /app/server/data and set the ENCRYPTION_KEY via Secrets.

Security Best Practices

• Network Isolation: Do not expose port 3001 to the public internet. Keep the service behind your LAN firewall or within a VPN. The React dashboard has no built‑in authentication layer.
• Environment Isolation: Always set NODE_ENV=production and avoid DEV_MODE=true, which enables insecure fallback keys.
• Backup Strategy: Configure FREEAPI_DB_BACKUP_PATH or FREEAPI_DB_BACKUP_URL to schedule automated SQLite backups. Preserve the same ENCRYPTION_KEY across restores and upgrades to maintain access to encrypted provider credentials.

Summary

• FreeLLMAPI aggregates free LLM providers behind a single OpenAI‑compatible /v1 endpoint, implemented in server/src/index.ts and server/src/services/router.ts.
• Encryption is mandatory: generate a 32‑byte hex ENCRYPTION_KEY and store it in your .env file; provider keys are encrypted using AES‑256‑GCM in server/src/db/model-pricing.ts.
• Persistence: SQLite data lives in /app/server/data; mount a named Docker volume to ensure data survives container restarts.
• Deployment: Use the official image ghcr.io/tashfeenahmed/freellmapi:latest with Docker Compose, binding to 0.0.0.0 for LAN access.
• Scaling: Share the SQLite volume and encryption key across lightweight replicas; the service consumes approximately 40 MB of memory per instance.

Frequently Asked Questions

What is the minimum environment setup to run FreeLLMAPI in production?

You need a Docker container runtime, a persistent volume mounted at /app/server/data, and a 64‑character hex ENCRYPTION_KEY set as an environment variable. The PORT defaults to 3001, and you should set HOST_BIND=0.0.0.0 to accept external connections. No external database is required; the service uses SQLite by default.

How do I backup the FreeLLMAPI database in production?

Set the environment variables FREEAPI_DB_BACKUP_PATH to a writable directory or FREEAPI_DB_BACKUP_URL to an HTTP endpoint that accepts POST requests. The service automatically writes encrypted SQLite backups according to the schedule defined in the health service. Always archive the ENCRYPTION_KEY alongside your backups, or the stored provider credentials will be unrecoverable.

Can I run FreeLLMAPI without Docker?

Yes, you can run the Node.js server directly using Node 20+. Install dependencies, build the React dashboard in the client/ directory, and start the server from server/src/index.ts. However, Docker is strongly recommended for production to ensure consistent environments, proper volume management for the SQLite database, and secure secret handling via environment files.

How does the automatic fallback mechanism work when a provider rate‑limits me?

The router in server/src/services/router.ts maintains a priority list of models. When a request receives a 429 or 5xx error, the server/src/services/ratelimit.ts module marks the key as rate‑limited and the router immediately retries the request against the next healthy model in the chain. This happens transparently to the client, though you can inspect the x-routed-via response header to see which provider ultimately served the request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}