# How to Use FreeLLMAPI for Local Development: Complete Setup Guide

> Learn how to use FreeLLMAPI for local development. This guide shows you how to set up this OpenAI-compatible proxy to aggregate free LLM tiers on your local machine.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-26

---

**FreeLLMAPI is an OpenAI-compatible proxy that aggregates free tiers from dozens of LLM providers into a single local stack, running as a Node.js 20+ Express server with a bundled React dashboard that requires only a generated encryption key to start.**

FreeLLMAPI, maintained in the `tashfeenahmed/freellmapi` repository, lets you route requests across multiple free-tier providers through a unified API endpoint. When running FreeLLMAPI for local development, you get a full-stack environment with an Express proxy on port 3001 and a Vite-powered React dashboard on port 5173. This guide covers the exact steps to clone, configure, and start sending requests using the local development server.

## Architecture Overview

FreeLLMAPI consists of several interconnected components:

- **Express proxy** – Receives OpenAI-style requests on `/:v1/*` and forwards them to the router. Entry point is [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts).
- **Router** – Picks the highest-priority model that has a healthy key and is under its rate limits, decrypts the stored key, and calls the provider SDK. Implemented in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).
- **Rate-limit ledger** – In-memory RPM/RPD/TPM/TPD counters backed by SQLite; applies cooldowns on 429/5xx responses. Found in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts).
- **Health service** – Periodically probes each key and marks it `healthy`, `rate_limited`, `invalid`, or `error`. Located at [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts).
- **Provider adapters** – One file per upstream provider (Google, Groq, Mistral, etc.) implementing a common `Provider` interface. Stored in `server/src/providers/*.ts`.
- **Dashboard** – React + Vite UI for managing keys and inspecting analytics. Entry point is [`client/src/main.tsx`](https://github.com/tashfeenahmed/freellmapi/blob/main/client/src/main.tsx).
- **Encrypted key storage** – SQLite stores API keys encrypted with AES-256-GCM; the encryption key is supplied via `.env`. Database layer in `server/src/db/*.ts` uses `better-sqlite3`.

## Prerequisites

Before starting, ensure you have:

- **Node.js 20+** installed
- **npm** package manager
- A Unix-like terminal (Linux, macOS, or WSL) for the key generation commands

## Step-by-Step Local Setup

### Clone and Install Dependencies

Clone the repository and install both server and client workspace dependencies:

```bash
git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
npm install

```

### Generate the Encryption Key

The server requires a 256-bit encryption key to secure provider credentials. Generate and write it to `.env`:

```bash
cp .env.example .env
ENCRYPTION_KEY="$(node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))')"
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env

```

The `ENCRYPTION_KEY` variable is mandatory for the AES-256-GCM implementation in `server/src/db/*.ts` to encrypt and decrypt stored provider credentials.

### Start Development Servers

Launch both services simultaneously:

```bash
npm run dev

```

This command starts the Express proxy on `localhost:3001` and the Vite dev server for the dashboard on `localhost:5173`.

### Add Provider Keys

Open `http://localhost:5173` in your browser, navigate to the **Keys** page, and paste each provider’s API key. The React dashboard stores these encrypted in the local SQLite database.

### Retrieve the Unified API Key

After adding at least one provider, the dashboard header displays a `freellmapi-...` bearer token. This unified token is what your client libraries use for all requests.

## Making API Calls

Point any OpenAI-compatible client at `http://localhost:3001/v1` with the unified token. The router automatically selects a free model, handles fallbacks on rate-limit errors, and injects an `X-Routed-Via` header indicating the actual upstream provider.

### Python Client Example

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))  # e.g., "google/gemini-2.5-flash"

```

### cURL Example

```bash
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "hi"}]
      }'

```

### Streaming and Embeddings

```python

# Streaming response

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

# Embeddings request

emb = client.embeddings.create(
    model="auto",
    input=["the quick brown fox", "pack my box with five dozen liquor jugs"],
)
print(len(emb.data), "vectors of", len(emb.data[0].embedding), "dims")

```

## Core Routing Logic

Understanding three critical services helps debug local behavior:

**Router** ([`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)) – Implements model selection, key decryption, and provider invocation. It prioritizes keys based on your dashboard configuration and health status.

**Rate Limiting** ([`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts)) – Tracks per-key RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day). When limits are exceeded, the router marks the key as rate-limited and attempts the next provider in the chain.

**Health Service** ([`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts)) – Runs periodic probes against each stored key, updating status to `healthy`, `rate_limited`, `invalid`, or `error` based on provider responses.

## LAN Access

For development across multiple devices on your local network, run:

```bash
npm run dev:lan

```

This passes `--host` to the Vite dev server, exposing the dashboard on a network-accessible URL printed in the terminal output.

## Summary

- FreeLLMAPI requires Node.js 20+ and runs two services locally: Express on `:3001` and the React dashboard on `:5173`
- You must generate an `ENCRYPTION_KEY` in `.env` for AES-256-GCM credential storage in `server/src/db/*.ts`
- Provider keys are added via the dashboard and stored encrypted in SQLite via `better-sqlite3`
- The unified `freellmapi-...` token routes requests through [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) with automatic fallback across providers
- Run `npm run dev` for local development or `npm run dev:lan` for network access

## Frequently Asked Questions

### What file contains the Express server entry point?

The Express proxy entry point is [`server/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/index.ts). This file initializes the server to receive OpenAI-style requests on `/:v1/*` and forwards them to the router service.

### How does FreeLLMAPI handle encryption locally?

The system uses AES-256-GCM encryption implemented in `server/src/db/*.ts` using the `better-sqlite3` library. The encryption key must be supplied through the `ENCRYPTION_KEY` environment variable in your `.env` file to decrypt provider credentials at runtime.

### Why does my request include an X-Routed-Via header?

The `X-Routed-Via` header is injected by the router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) to indicate which upstream provider actually processed the request. This helps you debug which free tier is being consumed and verify fallback behavior.

### Can I use FreeLLMAPI with existing OpenAI client libraries?

Yes. FreeLLMAPI is fully OpenAI-compatible. Configure your client with `base_url="http://localhost:3001/v1"` and the unified `freellmapi-...` API key from the dashboard. This allows you to use the standard OpenAI SDK without code changes while routing through the free tier pool.