How to Set Up FreeLLMAPI Locally: Complete Self-Hosting Guide

You can set up FreeLLMAPI locally by cloning the tashfeenahmed/freellmapi repository, copying .env.example to .env with your provider API keys, and running docker compose up --build to start the gateway, then using the CLI doctor utility or curl to verify the health endpoint.

FreeLLMAPI is a self-hosted LLM gateway that centralizes requests to multiple providers while managing rate limits and fallback logic. Setting up FreeLLMAPI locally requires Docker for the server component and Node.js for the CLI and client utilities. The repository organizes functionality into three distinct parts—Server, CLI, and Client—each serving a specific role in the request lifecycle.

Repository Architecture and Key Components

Before installing, understand how the tashfeenahmed/freellmapi codebase is organized. The project splits responsibilities across three logical units:

• Server: Located in server/src/, handles HTTP API routing, quota management, and model discovery. The core logic resides in server/src/services/router.ts, which implements provider selection and rate-limiting.
• CLI: Found in cli/src/, provides diagnostic and configuration utilities. Entry point is cli/src/index.ts, with health checks implemented in cli/src/doctor.ts.
• Client: A minimal testing UI in client/index.html and related files, used for interactive API testing.

Step-by-Step FreeLLMAPI Local Setup

Clone the Repository and Configure Environment Variables

Start by cloning the repository and preparing your environment file. The project requires API keys from providers like OpenRouter or Groq:

git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
cp .env.example .env

Edit .env to add your provider keys. This file defines required variables such as OPENROUTER_API_KEY and optional configuration flags that the router consumes.

Build and Launch the Docker Container

The server runs inside Docker. According to docker/README.md, use Docker Compose to build and start the service:

docker compose up --build

Alternatively, build the image manually with docker build . -t freellmapi and run it with port mappings. This step initializes the HTTP API on port 3000 and loads the provider configurations from your .env file.

Install Node Dependencies for CLI and Client

While the server runs in Docker, the CLI and client require local Node.js dependencies. Run npm ci at the repository root to install workspace dependencies, then install specific dependencies for the CLI and client components:

npm ci
cd cli && npm ci
cd ../client && npm ci

Start the Development Environment

For a unified development experience, use the bootstrap script defined in scripts/dev-bootstrap.sh. Execute:

npm run dev

This command orchestrates the Docker container, Node processes, and client development server, typically exposing the client UI on port 5173 and the API on port 3000.

Verify Your Local Installation

Confirm the setup using the built-in diagnostic tool or a direct HTTP request. The CLI doctor utility (cli/src/doctor.ts) checks Docker status, environment variables, and provider connectivity:

cd cli
node src/index.js doctor

Expected output includes verification that Docker is running, the .env file exists, and required provider keys are present. Alternatively, verify the server directly:

curl http://localhost:3000/health

Testing with the Client and Direct API Calls

Once verified, test the gateway functionality using either the web client or command-line requests.

To launch the interactive testing UI:

cd client
npm run dev

Open http://localhost:5173 to select models and submit prompts through the browser.

For programmatic verification, send a request to the local API endpoint defined in server/src/services/router.ts:

curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Explain why the sky is blue"}]
      }'

Summary

• FreeLLMAPI requires Docker for the server and Node.js for CLI/client components.
• Configuration happens via .env file copied from .env.example, containing provider API keys.
• Run docker compose up --build to start the gateway server on port 3000.
• Use node cli/src/index.js doctor from cli/src/doctor.ts to diagnose setup issues.
• The client UI (client/index.html) and direct curl requests both validate that server/src/services/router.ts is correctly routing to LLM providers.

Frequently Asked Questions

What are the system requirements for running FreeLLMAPI locally?

You need Docker Engine 20.10+ and Node.js 18+ installed on your machine. The server containerizes all runtime dependencies, while the CLI and client require npm for dependency management.

How do I add new provider API keys after initial setup?

Edit the .env file in the repository root with the new keys, then restart the Docker container with docker compose restart. The server/src/services/router.ts reloads environment variables on startup.

Can I run FreeLLMAPI without Docker?

While the CLI and client run directly via Node.js, the server component is designed specifically for containerized deployment as documented in docker/README.md. Running it outside Docker requires manual installation of dependencies defined in the server configuration.

Where can I find detailed troubleshooting steps?

Consult docs/install.md for extended installation guidance. For runtime diagnostics, execute the CLI doctor command (node cli/src/index.js doctor), which validates Docker status, environment file presence, and provider connectivity according to the logic in cli/src/doctor.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}