# How to Set Up FreeLLMAPI Locally: Complete Self-Hosting Guide

> Learn how to set up FreeLLMAPI locally with this complete self-hosting guide. Clone the repo, configure your keys, and launch the gateway using Docker for seamless local LLM API access.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-26

---

**You can set up FreeLLMAPI locally by cloning the tashfeenahmed/freellmapi repository, copying `.env.example` to `.env` with your provider API keys, and running `docker compose up --build` to start the gateway, then using the CLI doctor utility or curl to verify the health endpoint.**

FreeLLMAPI is a self-hosted LLM gateway that centralizes requests to multiple providers while managing rate limits and fallback logic. Setting up FreeLLMAPI locally requires Docker for the server component and Node.js for the CLI and client utilities. The repository organizes functionality into three distinct parts—**Server**, **CLI**, and **Client**—each serving a specific role in the request lifecycle.

## Repository Architecture and Key Components

Before installing, understand how the tashfeenahmed/freellmapi codebase is organized. The project splits responsibilities across three logical units:

- **Server**: Located in `server/src/`, handles HTTP API routing, quota management, and model discovery. The core logic resides in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), which implements provider selection and rate-limiting.
- **CLI**: Found in `cli/src/`, provides diagnostic and configuration utilities. Entry point is [`cli/src/index.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/index.ts), with health checks implemented in [`cli/src/doctor.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/doctor.ts).
- **Client**: A minimal testing UI in [`client/index.html`](https://github.com/tashfeenahmed/freellmapi/blob/main/client/index.html) and related files, used for interactive API testing.

## Step-by-Step FreeLLMAPI Local Setup

### Clone the Repository and Configure Environment Variables

Start by cloning the repository and preparing your environment file. The project requires API keys from providers like OpenRouter or Groq:

```bash
git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
cp .env.example .env

```

Edit `.env` to add your provider keys. This file defines required variables such as `OPENROUTER_API_KEY` and optional configuration flags that the router consumes.

### Build and Launch the Docker Container

The server runs inside Docker. According to [`docker/README.md`](https://github.com/tashfeenahmed/freellmapi/blob/main/docker/README.md), use Docker Compose to build and start the service:

```bash
docker compose up --build

```

Alternatively, build the image manually with `docker build . -t freellmapi` and run it with port mappings. This step initializes the HTTP API on port 3000 and loads the provider configurations from your `.env` file.

### Install Node Dependencies for CLI and Client

While the server runs in Docker, the CLI and client require local Node.js dependencies. Run `npm ci` at the repository root to install workspace dependencies, then install specific dependencies for the CLI and client components:

```bash
npm ci
cd cli && npm ci
cd ../client && npm ci

```

### Start the Development Environment

For a unified development experience, use the bootstrap script defined in [`scripts/dev-bootstrap.sh`](https://github.com/tashfeenahmed/freellmapi/blob/main/scripts/dev-bootstrap.sh). Execute:

```bash
npm run dev

```

This command orchestrates the Docker container, Node processes, and client development server, typically exposing the client UI on port 5173 and the API on port 3000.

## Verify Your Local Installation

Confirm the setup using the built-in diagnostic tool or a direct HTTP request. The **CLI doctor utility** ([`cli/src/doctor.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/doctor.ts)) checks Docker status, environment variables, and provider connectivity:

```bash
cd cli
node src/index.js doctor

```

Expected output includes verification that Docker is running, the `.env` file exists, and required provider keys are present. Alternatively, verify the server directly:

```bash
curl http://localhost:3000/health

```

## Testing with the Client and Direct API Calls

Once verified, test the gateway functionality using either the web client or command-line requests.

To launch the interactive testing UI:

```bash
cd client
npm run dev

```

Open `http://localhost:5173` to select models and submit prompts through the browser.

For programmatic verification, send a request to the local API endpoint defined in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts):

```bash
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Explain why the sky is blue"}]
      }'

```

## Summary

- FreeLLMAPI requires **Docker** for the server and **Node.js** for CLI/client components.
- Configuration happens via `.env` file copied from `.env.example`, containing provider API keys.
- Run `docker compose up --build` to start the gateway server on port 3000.
- Use `node cli/src/index.js doctor` from [`cli/src/doctor.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/doctor.ts) to diagnose setup issues.
- The client UI ([`client/index.html`](https://github.com/tashfeenahmed/freellmapi/blob/main/client/index.html)) and direct curl requests both validate that [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) is correctly routing to LLM providers.

## Frequently Asked Questions

### What are the system requirements for running FreeLLMAPI locally?

You need Docker Engine 20.10+ and Node.js 18+ installed on your machine. The server containerizes all runtime dependencies, while the CLI and client require npm for dependency management.

### How do I add new provider API keys after initial setup?

Edit the `.env` file in the repository root with the new keys, then restart the Docker container with `docker compose restart`. The [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) reloads environment variables on startup.

### Can I run FreeLLMAPI without Docker?

While the CLI and client run directly via Node.js, the server component is designed specifically for containerized deployment as documented in [`docker/README.md`](https://github.com/tashfeenahmed/freellmapi/blob/main/docker/README.md). Running it outside Docker requires manual installation of dependencies defined in the server configuration.

### Where can I find detailed troubleshooting steps?

Consult [`docs/install.md`](https://github.com/tashfeenahmed/freellmapi/blob/main/docs/install.md) for extended installation guidance. For runtime diagnostics, execute the CLI doctor command (`node cli/src/index.js doctor`), which validates Docker status, environment file presence, and provider connectivity according to the logic in [`cli/src/doctor.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/cli/src/doctor.ts).