How to Use FreeLLMAPI for Local Development: Complete Setup Guide

FreeLLMAPI is an OpenAI-compatible proxy that aggregates free tiers from dozens of LLM providers into a single local stack, running as a Node.js 20+ Express server with a bundled React dashboard that requires only a generated encryption key to start.

FreeLLMAPI, maintained in the tashfeenahmed/freellmapi repository, lets you route requests across multiple free-tier providers through a unified API endpoint. When running FreeLLMAPI for local development, you get a full-stack environment with an Express proxy on port 3001 and a Vite-powered React dashboard on port 5173. This guide covers the exact steps to clone, configure, and start sending requests using the local development server.

Architecture Overview

FreeLLMAPI consists of several interconnected components:

• Express proxy – Receives OpenAI-style requests on /:v1/* and forwards them to the router. Entry point is server/src/index.ts.
• Router – Picks the highest-priority model that has a healthy key and is under its rate limits, decrypts the stored key, and calls the provider SDK. Implemented in server/src/services/router.ts.
• Rate-limit ledger – In-memory RPM/RPD/TPM/TPD counters backed by SQLite; applies cooldowns on 429/5xx responses. Found in server/src/services/ratelimit.ts.
• Health service – Periodically probes each key and marks it healthy, rate_limited, invalid, or error. Located at server/src/services/health.ts.
• Provider adapters – One file per upstream provider (Google, Groq, Mistral, etc.) implementing a common Provider interface. Stored in server/src/providers/*.ts.
• Dashboard – React + Vite UI for managing keys and inspecting analytics. Entry point is client/src/main.tsx.
• Encrypted key storage – SQLite stores API keys encrypted with AES-256-GCM; the encryption key is supplied via .env. Database layer in server/src/db/*.ts uses better-sqlite3.

Prerequisites

Before starting, ensure you have:

• Node.js 20+ installed
• npm package manager
• A Unix-like terminal (Linux, macOS, or WSL) for the key generation commands

Step-by-Step Local Setup

Clone and Install Dependencies

Clone the repository and install both server and client workspace dependencies:

git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
npm install

Generate the Encryption Key

The server requires a 256-bit encryption key to secure provider credentials. Generate and write it to .env:

cp .env.example .env
ENCRYPTION_KEY="$(node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))')"
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env

The ENCRYPTION_KEY variable is mandatory for the AES-256-GCM implementation in server/src/db/*.ts to encrypt and decrypt stored provider credentials.

Start Development Servers

Launch both services simultaneously:

npm run dev

This command starts the Express proxy on localhost:3001 and the Vite dev server for the dashboard on localhost:5173.

Add Provider Keys

Open http://localhost:5173 in your browser, navigate to the Keys page, and paste each provider’s API key. The React dashboard stores these encrypted in the local SQLite database.

Retrieve the Unified API Key

After adding at least one provider, the dashboard header displays a freellmapi-... bearer token. This unified token is what your client libraries use for all requests.

Making API Calls

Point any OpenAI-compatible client at http://localhost:3001/v1 with the unified token. The router automatically selects a free model, handles fallbacks on rate-limit errors, and injects an X-Routed-Via header indicating the actual upstream provider.

Python Client Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))  # e.g., "google/gemini-2.5-flash"

cURL Example

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "hi"}]
      }'

Streaming and Embeddings

# Streaming response

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

# Embeddings request

emb = client.embeddings.create(
    model="auto",
    input=["the quick brown fox", "pack my box with five dozen liquor jugs"],
)
print(len(emb.data), "vectors of", len(emb.data[0].embedding), "dims")

Core Routing Logic

Understanding three critical services helps debug local behavior:

Router (server/src/services/router.ts) – Implements model selection, key decryption, and provider invocation. It prioritizes keys based on your dashboard configuration and health status.

Rate Limiting (server/src/services/ratelimit.ts) – Tracks per-key RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day). When limits are exceeded, the router marks the key as rate-limited and attempts the next provider in the chain.

Health Service (server/src/services/health.ts) – Runs periodic probes against each stored key, updating status to healthy, rate_limited, invalid, or error based on provider responses.

LAN Access

For development across multiple devices on your local network, run:

npm run dev:lan

This passes --host to the Vite dev server, exposing the dashboard on a network-accessible URL printed in the terminal output.

Summary

• FreeLLMAPI requires Node.js 20+ and runs two services locally: Express on :3001 and the React dashboard on :5173
• You must generate an ENCRYPTION_KEY in .env for AES-256-GCM credential storage in server/src/db/*.ts
• Provider keys are added via the dashboard and stored encrypted in SQLite via better-sqlite3
• The unified freellmapi-... token routes requests through server/src/services/router.ts with automatic fallback across providers
• Run npm run dev for local development or npm run dev:lan for network access

Frequently Asked Questions

What file contains the Express server entry point?

The Express proxy entry point is server/src/index.ts. This file initializes the server to receive OpenAI-style requests on /:v1/* and forwards them to the router service.

How does FreeLLMAPI handle encryption locally?

The system uses AES-256-GCM encryption implemented in server/src/db/*.ts using the better-sqlite3 library. The encryption key must be supplied through the ENCRYPTION_KEY environment variable in your .env file to decrypt provider credentials at runtime.

Why does my request include an X-Routed-Via header?

The X-Routed-Via header is injected by the router in server/src/services/router.ts to indicate which upstream provider actually processed the request. This helps you debug which free tier is being consumed and verify fallback behavior.

Can I use FreeLLMAPI with existing OpenAI client libraries?

Yes. FreeLLMAPI is fully OpenAI-compatible. Configure your client with base_url="http://localhost:3001/v1" and the unified freellmapi-... API key from the dashboard. This allows you to use the standard OpenAI SDK without code changes while routing through the free tier pool.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}