# How to Set Up OmniRoute as a Local AI Gateway: Complete Self-Hosted Setup Guide

> Set up OmniRoute as your local AI gateway with this self-hosted guide. Connect to 351 providers via OpenAI-compatible endpoints on localhost:20128. Start your zero-configuration AI journey today.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-30

---

**OmniRoute is a self-hosted AI gateway that exposes OpenAI-compatible endpoints on `localhost:20128` with zero-configuration setup, automatically routing LLM requests across 351 providers through a Next.js 16 streaming architecture.**

OmniRoute (diegosouzapw/OmniRoute) operates as a local AI gateway that aggregates dozens of LLM providers behind a single unified interface. Unlike cloud-based solutions, this open-source tool runs entirely on your infrastructure, offering complete control over request routing, provider selection, and data privacy. The system requires no API keys out of the box, utilizing the free "OpenCode" provider to handle initial requests immediately after installation.

## Prerequisites and Installation

You can install OmniRoute via npm or Docker. The npm package provides a global CLI command, while the Docker image offers containerized deployment.

**NPM Installation:**

```bash
npm install -g omniroute

```

**Docker Installation:**

```bash
docker run -p 20128:20128 diegosouzapw/omniroute

```

The gateway requires Node.js 16+ for native execution and exposes port `20128` by default.

## Architecture Overview

OmniRoute's architecture centers on a Next.js 16 application that processes HTTP requests through a modular pipeline. The system separates concerns between API routing, request handling, provider execution, and response translation.

### Request Flow Pipeline

When you send a request to `http://localhost:20128/v1/chat/completions`, OmniRoute processes it through five distinct stages:

1. **API Entry Point** – Routes in `src/app/api/v1/*` validate payloads using Zod schemas before forwarding to handlers.
2. **Core Handler** – [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) (via `handleChatCore`) orchestrates the request pipeline and applies policy checks.
3. **Combo Routing** – [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) implements 19 routing strategies (including auto-combo) to select optimal providers.
4. **Provider Execution** – Executor modules in `open-sse/executors/` handle HTTP communication with upstream LLM services.
5. **Response Translation** – Translators in `open-sse/translator/` normalize provider-specific payloads into the OpenAI schema before streaming via SSE.

### Key Source Files

Understanding these critical files helps with customization and debugging:

- **`bin/omniroute.mjs`** – Server entry point that initializes the Next.js application and loads gateway configurations.
- **[`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts)** – Receives OpenAI-compatible POST requests and manages the initial request validation.
- **[`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts)** – Contains `handleChatCore` function that manages the complete request lifecycle.
- **[`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts)** – Implements the combo routing engine supporting strategies like auto-combo and fill-first.
- **`open-sse/executors/*`** – Directory containing provider-specific HTTP logic for each supported LLM service.
- **`open-sse/translator/*`** – Modules that transform diverse provider response formats into standard OpenAI-compatible streams.
- **[`docs/reference/PROVIDER_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/PROVIDER_REFERENCE.md)** – Catalog documenting all 351 available providers and their free-tier availability.
- **`src/app/(dashboard)/**`** – Web interface components for monitoring quotas, provider status, and routing policies.

## Starting the Gateway

Launching OmniRoute requires a single command after installation. The server entry point in `bin/omniroute.mjs` automatically configures the environment and binds to the default port.

```bash
omniroute

```

The service starts on **http://localhost:20128** and immediately begins accepting requests. The **auto-combo** strategy activates automatically, including the free "OpenCode" provider so your first request succeeds without API keys.

To verify the server is running:

```bash
curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

```

## Making Requests to Local Endpoints

OmniRoute accepts standard OpenAI SDK calls by changing the base URL to your local gateway. All compatible clients can point at `http://localhost:20128/v1` and issue normal `/chat/completions` calls.

**cURL Example:**

```bash
curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Explain recursion"}]}'

```

**Python OpenAI SDK Example:**

```python
import openai

openai.api_base = "http://localhost:20128/v1"
openai.api_key = "unused"  # Required by SDK but ignored by gateway

response = openai.ChatCompletion.create(
    model="auto",
    messages=[{"role": "user", "content": "Explain the Fibonacci sequence"}]
)
print(response.choices[0].message.content)

```

The `model: "auto"` parameter triggers the combo router in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) to intelligently select between available providers based on cost, latency, and availability.

## Configuring Providers and Routing

While OmniRoute works zero-config using the built-in OpenCode provider, you can customize the 351-provider catalog documented in [`docs/reference/PROVIDER_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/PROVIDER_REFERENCE.md).

**Accessing the Dashboard:**

Navigate to `http://localhost:20128/dashboard/providers` to view the web UI implemented in `src/app/(dashboard)/`. This interface allows you to enable, disable, or prioritize specific providers without editing configuration files.

**Adding Custom Providers:**

For enterprise or private endpoints, add providers programmatically:

```bash
curl -X POST http://localhost:20128/api/v1/providers \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-custom-provider",
    "apiKey": "sk-your-key-here",
    "baseUrl": "https://api.custom-provider.com/v1"
  }'

```

## Advanced Routing Strategies

The combo service in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) implements 19 distinct routing strategies beyond the default auto-combo. These include fill-first (prioritizing cheapest available), round-robin (load balancing), and failover (cascading through providers on error).

To specify a strategy, include it in your request parameters or configure it as the default in the dashboard. The handler in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) evaluates these policies before executing provider calls.

## Summary

- OmniRoute provides a **zero-config local AI gateway** at `localhost:20128` with OpenAI-compatible endpoints.
- The architecture uses **Next.js 16** for API routing (`src/app/api/v1/*`) and **open-sse** modules for streaming execution.
- **Auto-combo routing** automatically includes the free OpenCode provider, enabling immediate testing without API keys.
- The system supports **351 providers** cataloged in [`docs/reference/PROVIDER_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/PROVIDER_REFERENCE.md) with configurable strategies via [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts).
- Both **npm** (`npm i -g omniroute`) and **Docker** deployment options require only port 20128 exposure to begin serving requests.

## Frequently Asked Questions

### What port does OmniRoute use by default?

OmniRoute binds to **port 20128** by default. The server entry point in `bin/omniroute.mjs` initializes the Next.js application on this port, making the gateway accessible at `http://localhost:20128`. You can map this to any host port when using Docker (`-p 20128:20128`).

### Does OmniRoute require API keys to start?

No. OmniRoute operates **zero-config** on first launch by automatically enabling the free "OpenCode" provider through the auto-combo strategy implemented in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts). This allows immediate testing of the `/v1/chat/completions` endpoint without configuring credentials.

### How does OmniRoute handle different LLM provider formats?

OmniRoute normalizes responses through a dedicated translation layer located in `open-sse/translator/`. After executor modules in `open-sse/executors/` fetch responses from upstream providers, translators convert proprietary schemas into the standard OpenAI-compatible format before streaming results to clients via Server-Sent Events (SSE).

### Can I use standard OpenAI client libraries with OmniRoute?

Yes. Any OpenAI-compatible client library works by changing the `api_base` or `base_url` parameter to `http://localhost:20128/v1`. The API routes in [`src/app/api/v1/chat/completions/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/chat/completions/route.ts) accept standard request payloads, allowing drop-in replacement of OpenAI's official endpoint with your local gateway.