# How to Get Started with FreeLLM API: A Self-Hosted OpenAI-Compatible Proxy

> Get started with FreeLLM API a self hosted OpenAI compatible proxy Use free LLM tiers seamlessly with automatic routing and fallback handling

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: getting-started
- Published: 2026-07-02

---

**FreeLLM API is a self-hosted, OpenAI-compatible proxy that aggregates free LLM tiers into a single `/v1` endpoint with automatic routing and fallback handling.**

FreeLLM API (freellmapi) is an open-source project by tashfeenahmed that unifies dozens of LLM providers behind one OpenAI-style interface. The system is built on Node.js/TypeScript and handles provider selection, rate limit management, and encrypted credential storage automatically.

## Architecture Overview

The system centers on a **router** that intelligently distributes requests across available free-tier providers.

### Core Components

**Express Server** – The entry point defined in [`server/src/app.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/app.ts) receives OpenAI-style requests at `/v1/*` and mounts the admin dashboard.

**Router Service** – Implemented in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), this component selects the highest-priority healthy model, injects context-handoff messages when switching providers, and retries up to 20 fall-over attempts during failures.

**Rate-Limit Ledger** – Located in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), this maintains in-memory counters backed by SQLite to enforce per-platform RPM/RPD/TPM/TPD limits and manages cooldowns after HTTP 429 or 5xx responses.

**Provider Adapters** – Files in `server/src/providers/*.ts` (such as the Google adapter) implement `chatCompletion()` and `streamChatCompletion()` for each vendor, translating wire-format differences like Anthropic's message structure.

**Health Service** – The [`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts) module performs periodic probes of stored keys, marking them as `healthy`, `rate_limited`, `invalid`, or `error` for the router to filter.

**Encrypted Storage** – Database operations in [`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts) use AES-256-GCM encryption for API keys, with the encryption key supplied via environment variables.

**Dashboard UI** – A React + Vite admin panel in [`client/src/App.tsx`](https://github.com/tashfeenahmed/freellmapi/blob/main/client/src/App.tsx) provides key management, analytics, and token generation.

**Desktop Wrapper** – An Electron application in [`desktop/src/main.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/desktop/src/main.ts) runs the server locally with a tray pop-over for live statistics.

The router syncs a signed model catalog from freellmapi.co twice daily, ensuring new free models appear automatically. Responses include `X-Routed-Via` headers identifying the provider and `X-Fallback-Attempts` headers when fall-over occurs.

## Installation Methods

### Server Installation

Clone the repository and install dependencies:

```bash
git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
npm install

```

Configure your environment using the template in `.env.example`:

```bash

# Required: 32-byte encryption key for AES-256-GCM

ENCRYPTION_KEY=your-32-byte-hex-key-here

# Optional: Server port (default 3001)

PORT=3001

```

Start the server:

```bash
npm run server

```

### Desktop Application

For a zero-install experience on macOS or Windows, build the Electron wrapper:

```bash

# macOS

npm run desktop:dist

# Windows

npm run desktop:dist:win

```

The desktop application runs the same Node.js server locally while providing a system tray interface for monitoring live stats.

## Using the API

FreeLLM API exposes an OpenAI-compatible endpoint at `http://localhost:3001/v1`. Use the unified API key generated in the dashboard (format: `freellmapi-your-unified-key`).

### Python OpenAI SDK

Configure the OpenAI client to point to your local instance:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",  # Let the router choose the best available model

    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)

print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

```

### cURL Requests

Send requests directly via HTTP:

```bash
curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "hi"}]
      }'

```

### Streaming Responses

Enable streaming for real-time token delivery:

```python
stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream me a haiku about SQLite."}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

```

### VS Code Integration

Configure the Continue extension for ghost-text autocomplete:

```yaml
models:
  - name: FreeLLMAPI Autocomplete
    provider: openai
    model: auto
    apiBase: http://localhost:3001/v1
    apiKey: freellmapi-your-unified-key
    useLegacyCompletionsEndpoint: true
    roles:
      - autocomplete

```

## Key Source Files Reference

Understanding these files helps customize routing behavior:

- **[`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)** – Core routing logic, fallback chain management, and context handoff injection when switching models mid-conversation.

- **[`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts)** – In-memory rate-limit counters and cooldown handling for provider-specific quotas.

- **`server/src/providers/*.ts`** – Individual adapter implementations for each LLM vendor (Google, Groq, Cerebras, OpenRouter) handling translation between FreeLLM API wire format and vendor SDKs.

- **[`server/src/services/health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/health.ts)** – Periodic health checks that update key status to prevent routing to failed providers.

- **[`server/src/app.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/app.ts)** – Express server setup, route mounting for `/v1/*` endpoints, and admin UI serving.

- **[`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts)** – Encrypted SQLite storage handling using AES-256-GCM envelopes.

- **`client/src/`** – React/Vite dashboard source for key management and analytics visualization.

- **[`desktop/src/main.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/desktop/src/main.ts)** – Electron main process that wraps the server for standalone desktop execution.

## Summary

- **FreeLLM API** aggregates free-tier LLM providers into one OpenAI-compatible endpoint hosted at `localhost:3001/v1`.

- The **router** in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) automatically selects healthy models and handles up to 20 fallback attempts when providers fail or hit rate limits.

- **Provider adapters** in `server/src/providers/*.ts` normalize requests across different vendor APIs, while the **health service** continuously monitors key validity.

- **Encrypted storage** uses AES-256-GCM via [`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts), requiring a 32-byte `ENCRYPTION_KEY` in your environment configuration.

- Available as a **Node.js server** for production deployments or an **Electron desktop app** for local use with system tray monitoring.

## Frequently Asked Questions

### What is the difference between the server and desktop versions?

The **server** version is a standard Node.js/Express application designed for production deployments or custom hosting setups. The **desktop** version packages the same server inside an Electron application, providing a native system tray interface and automatic local execution without requiring manual Node.js installation. Both use identical routing logic from [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).

### How does FreeLLM API handle rate limits from individual providers?

The system maintains an in-memory **rate-limit ledger** in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) that tracks requests per minute (RPM), requests per day (RPD), tokens per minute (TPM), and tokens per day (TPD) for each provider. When a provider returns HTTP 429 or 5xx errors, the service marks that key as rate-limited in the health status and enforces a cooldown period before retrying, automatically routing subsequent requests to the next available provider in the priority chain.

### Can I force requests to use a specific provider instead of "auto"?

While the documentation emphasizes using `"model": "auto"` to leverage the intelligent routing system, you can specify individual models if configured in your provider adapters. The router in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) selects from available models based on the request parameters and health status. To use a specific provider, you would need to configure that provider's key in the dashboard and potentially modify the model name to match the specific provider's model identifier in your request.

### Is my API key data secure when using FreeLLM API?

Yes. According to the source code in [`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts), all provider API keys are stored using **AES-256-GCM encryption** in a local SQLite database. The encryption key is never stored in the database itself; you must supply it via the `ENCRYPTION_KEY` environment variable defined in `.env.example`. This ensures that even if the database file is compromised, the keys remain encrypted without access to your environment configuration.