# How to Run FreeLLMAPI in a Production Environment: Complete Deployment Guide

> Deploy FreeLLMAPI in production with Docker. Secure credentials with an encryption key and expose the service safely. Get the complete deployment guide.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-28

---

**Run FreeLLMAPI in a production environment by deploying the official Docker container with a persistent volume for SQLite, setting a 32‑hex `ENCRYPTION_KEY` for credential security, and exposing the service on your internal network via `HOST_BIND=0.0.0.0`.**

FreeLLMAPI is a self‑hosted, OpenAI‑compatible proxy developed by tashfeenahmed/freellmapi that aggregates dozens of free‑tier LLM providers behind a single `/v1` endpoint. When you run FreeLLMAPI in a production environment, you create a resilient gateway that automatically routes requests, handles rate limits, and encrypts provider keys at rest. This guide walks through the architecture, deployment steps, and configuration required for a secure, production‑ready installation.

## Architecture Components for Production

Understanding the core architecture helps you configure monitoring and troubleshoot issues when running FreeLLMAPI in production.

### Express Proxy and Request Routing

The entry point [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts) initializes an Express server that mounts the OpenAI‑compatible `/v1` routes. Incoming requests are forwarded to [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), which implements the core routing logic including model selection, sticky sessions, and automatic fallback chains. The router selects the best available model and invokes the appropriate provider adapter.

### Encrypted Provider Key Storage

Provider API credentials are never stored in plain text. The system uses **AES‑256‑GCM** encryption implemented in [`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts) and related database modules. The encryption key is derived from the `ENCRYPTION_KEY` environment variable, which must be a 64‑character hex string (32 bytes) generated via `openssl rand -hex 32`. Without this key, the service cannot decrypt stored credentials, making it the most critical secret for your production deployment.

### Rate Limiting and Persistence

Per‑key RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day) counters are maintained in memory and persisted to SQLite. The [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) module enforces these caps and triggers cooldown periods when providers return HTTP 429 or 5xx errors. By default, the SQLite database resides at `/app/server/data/freellmapi.db` inside the container.

### Health Monitoring and Fallback

A background health‑check service defined in [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) periodically probes each configured provider key to update status flags (`healthy`, `rate_limited`, `invalid`, `error`). When a request fails or hits a limit, the router automatically retries the next model in the fallback chain without client intervention.

## Production Deployment Steps

Follow these steps to deploy FreeLLMAPI securely in a production environment.

### 1. Pull the Official Docker Image

The repository provides a multi‑arch image that bundles the compiled Node.js server and the React/Vite dashboard.

```bash
docker pull ghcr.io/tashfeenahmed/freellmapi:latest

```

### 2. Generate an Encryption Key

Generate a cryptographically secure key and store it in your `.env` file:

```bash
echo "ENCRYPTION_KEY=$(openssl rand -hex 32)" >> .env

```

### 3. Configure Required Environment Variables

Create a `.env` file based on the repository’s `.env.example`. At minimum, specify:

- `ENCRYPTION_KEY` – The 64‑character hex string from step 2
- `PORT` – The internal port (default `3001`)
- `HOST_BIND` – Set to `0.0.0.0` to listen on all interfaces
- `FREEAPI_DB_PATH` – `/app/server/data/freellmapi.db`
- `NODE_ENV` – `production` (disables development fallbacks)

Optional variables include `REQUEST_ANALYTICS_RETENTION_DAYS` (default 90) and `FREEAPI_CONFIG_PATH` for declarative configuration.

### 4. Launch with Docker Compose

Use the official [`docker-compose.yml`](https://github.com/tashfeenahmed/freellmapi/blob/main/docker-compose.yml) to orchestrate the container with a persistent named volume:

```yaml
version: "3.8"
services:
  freellmapi:
    image: ghcr.io/tashfeenahmed/freellmapi:latest
    restart: unless-stopped
    ports:
      - "3001:3001"
    env_file: .env
    volumes:
      - freellmapi-data:/app/server/data
volumes:
  freellmapi-data:

```

Deploy with:

```bash
docker compose up -d

```

### 5. Add Provider Keys via the Dashboard

Once running, navigate to `http://<host>:3001` to access the dashboard. Navigate to the **Keys** page, paste each provider’s API key (e.g., Google, Groq), and save. The UI encrypts these credentials using your `ENCRYPTION_KEY` before writing them to SQLite.

### 6. Retrieve the Unified API Key

The dashboard header displays a unified token formatted as `freellmapi‑…`. Use this token as the `api_key` in any OpenAI‑compatible client. This is the only credential your applications need; individual provider keys remain encrypted and hidden.

## Production Configuration Examples

### Running the Container Directly

For environments where Docker Compose is unavailable, run the container manually:

```bash
docker run -d \
  --name freellmapi \
  --restart unless-stopped \
  -p 3001:3001 \
  -e ENCRYPTION_KEY="$(openssl rand -hex 32)" \
  -e PORT=3001 \
  -e HOST_BIND=0.0.0.0 \
  -e NODE_ENV=production \
  -v freellmapi-data:/app/server/data \
  ghcr.io/tashfeenahmed/freellmapi:latest

```

### Python Client Configuration

Connect your applications to the production endpoint using the unified key:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://my-prod-host:3001/v1",
    api_key="freellmapi-your-unified-key",
)

response = client.chat.completions.create(
    model="auto",  # Router selects the best available free model

    messages=[{"role": "user", "content": "Explain quantum computing"}],
)

print(response.choices[0].message.content)
print("Routed via:", response.headers.get("x-routed-via"))

```

### Testing with curl

Verify the deployment from the command line:

```bash
curl http://my-prod-host:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"What is the capital of France?"}]}'

```

## Scaling and Security Considerations

### Horizontal Scaling

The FreeLLMAPI process is lightweight (~40 MB RSS) and stateless except for the SQLite database. To scale horizontally:

1. Mount the same named volume (`freellmapi-data`) to all replicas, or migrate to an external database.
2. Ensure all replicas share the identical `ENCRYPTION_KEY`.
3. Use a load balancer to distribute requests across replicas.

For Kubernetes deployments, specify the volume claim for `/app/server/data` and set the `ENCRYPTION_KEY` via Secrets.

### Security Best Practices

- **Network Isolation**: Do not expose port `3001` to the public internet. Keep the service behind your LAN firewall or within a VPN. The React dashboard has no built‑in authentication layer.
- **Environment Isolation**: Always set `NODE_ENV=production` and avoid `DEV_MODE=true`, which enables insecure fallback keys.
- **Backup Strategy**: Configure `FREEAPI_DB_BACKUP_PATH` or `FREEAPI_DB_BACKUP_URL` to schedule automated SQLite backups. Preserve the same `ENCRYPTION_KEY` across restores and upgrades to maintain access to encrypted provider credentials.

## Summary

- **FreeLLMAPI** aggregates free LLM providers behind a single OpenAI‑compatible `/v1` endpoint, implemented in [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts) and [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).
- **Encryption** is mandatory: generate a 32‑byte hex `ENCRYPTION_KEY` and store it in your `.env` file; provider keys are encrypted using AES‑256‑GCM in [`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts).
- **Persistence**: SQLite data lives in `/app/server/data`; mount a named Docker volume to ensure data survives container restarts.
- **Deployment**: Use the official image `ghcr.io/tashfeenahmed/freellmapi:latest` with Docker Compose, binding to `0.0.0.0` for LAN access.
- **Scaling**: Share the SQLite volume and encryption key across lightweight replicas; the service consumes approximately 40 MB of memory per instance.

## Frequently Asked Questions

### What is the minimum environment setup to run FreeLLMAPI in production?

You need a Docker container runtime, a persistent volume mounted at `/app/server/data`, and a 64‑character hex `ENCRYPTION_KEY` set as an environment variable. The `PORT` defaults to `3001`, and you should set `HOST_BIND=0.0.0.0` to accept external connections. No external database is required; the service uses SQLite by default.

### How do I backup the FreeLLMAPI database in production?

Set the environment variables `FREEAPI_DB_BACKUP_PATH` to a writable directory or `FREEAPI_DB_BACKUP_URL` to an HTTP endpoint that accepts POST requests. The service automatically writes encrypted SQLite backups according to the schedule defined in the health service. Always archive the `ENCRYPTION_KEY` alongside your backups, or the stored provider credentials will be unrecoverable.

### Can I run FreeLLMAPI without Docker?

Yes, you can run the Node.js server directly using Node 20+. Install dependencies, build the React dashboard in the `client/` directory, and start the server from [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts). However, Docker is strongly recommended for production to ensure consistent environments, proper volume management for the SQLite database, and secure secret handling via environment files.

### How does the automatic fallback mechanism work when a provider rate‑limits me?

The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) maintains a priority list of models. When a request receives a 429 or 5xx error, the [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) module marks the key as rate‑limited and the router immediately retries the request against the next healthy model in the chain. This happens transparently to the client, though you can inspect the `x-routed-via` response header to see which provider ultimately served the request.