How to Set Up OmniRoute as a Self-Hosted AI Gateway: The Complete Installation Guide

You can set up OmniRoute as a self-hosted AI gateway by installing the global npm package (npm i -g omniroute), running omniroute run to start the server on port 20128, and sending requests to the OpenAI-compatible endpoint at http://localhost:20128/v1.

OmniRoute by diegosouzapw/OmniRoute is a self-hosted gateway that unifies 357 LLM providers behind a single OpenAI-compatible API. When run locally, it serves traffic through an intelligent Auto-Combo routing engine and requires no external API keys to begin processing requests. This guide walks you through the complete setup process from installation to production-ready configuration, referencing the actual source files that power each component.

Installation and Quick Start

Getting OmniRoute running locally takes less than five minutes. The gateway embeds all required services—including Redis and SQLite—so you do not need to configure external databases to begin.

Install the Global CLI

Run the following command to install the OmniRoute CLI and server components globally:

npm i -g omniroute

During installation, the script automatically generates a local .env file (ignored by Git) containing secrets for JWT and API-key encryption. This file is created according to the patterns defined in docs/DEVELOPER-ENVIRONMENT.md.

Start the Server

Launch the gateway with a single command:

omniroute run

The server initializes multiple embedded services:

• Next.js App Router under src/app/api/v1/ handles the OpenAI-compatible routes.
• MCP Server at open-sse/mcp-server/server.ts exposes 110 tools for advanced workflows.
• Embedded Infrastructure including Redis and local SQLite for caching and quota tracking, as documented in docs/frameworks/EMBEDDED-SERVICES.md.

Once started, the gateway listens on http://localhost:20128 and is ready to accept traffic immediately.

Verify with a Test Request

Confirm your installation by sending a request to the chat completions endpoint:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

Setting "model":"auto" invokes the Auto-Combo engine, which scores 19 routing strategies in real-time to select the best available provider. The response includes headers such as X-OmniRoute-Decision indicating which provider and strategy served the request, as referenced in docs/reference/API_REFERENCE.md.

Architecture Overview

Understanding the source code structure helps you customize and troubleshoot the gateway. OmniRoute uses a layered architecture where requests flow through distinct processing stages:

API Routing Layer

The Next.js App Router in src/app/api/v1/ serves as the entry point. Each route runs through a middleware stack handling CORS, Zod validation, optional authentication, and handler delegation.

Request Processing Layer

The open-sse/handlers/ directory contains the core processing logic. Specifically, open-sse/handlers/chatCore.ts orchestrates caching, rate limiting, and combo routing before dispatching to executors.

Provider Execution Layer

Provider-specific HTTP dispatchers live in open-sse/executors/. These executors handle the actual API calls to upstream services like OpenAI, Anthropic, or Google.

Protocol Translation Layer

The open-sse/translator/ directory manages format conversion between OpenAI, Claude, and Gemini wire protocols, ensuring compatibility regardless of the upstream provider.

Intelligence and Resilience

The Combo Service at open-sse/services/combo.ts implements the 19 routing strategies and the live 15-factor scoring engine. For fault tolerance, the src/lib/resilience/ directory contains the three resilience layers: provider circuit breakers, connection cooldowns, and model lockouts, all lazily refreshed to maintain traffic flow during upstream failures.

Configuring Providers and Authentication

While OmniRoute works immediately using free-tier providers, you can add proprietary API keys for higher quotas and premium models.

Using the Dashboard

Navigate to http://localhost:20128/dashboard and select Providers. Click Add Provider, choose from the catalog of 357 supported services, and paste your API key. The system encrypts and stores the key in the SQLite database via src/lib/db/providerKeys.ts.

Using the CLI

Configure providers programmatically through the interactive CLI:

omniroute configure openai

This command launches an interactive picker for setting keys and model preferences.

Database Storage

All configuration persists in a SQLite singleton accessed through src/lib/db/core.ts. The getDbInstance() function manages the connection, storing provider keys, usage statistics, and feature flags.

Key Source Files for Customization

When extending or debugging OmniRoute, these files contain the critical implementation details:

• src/app/api/v1/chat/completions/route.ts – Entry point for the /v1/chat/completions endpoint.
• open-sse/handlers/chatCore.ts – Orchestrates cache checks, rate limiting, and combo routing decisions.
• open-sse/services/combo.ts – Core Auto-Combo implementation with live scoring algorithms.
• src/lib/db/core.ts – SQLite database singleton used across all services.
• open-sse/mcp-server/server.ts – Launches the Model Context Protocol server providing 110 tools.
• src/lib/resilience/ – Circuit breaker and cooldown implementations documented in docs/architecture/RESILIENCE_GUIDE.md.

Advanced Configuration Options

Custom Routing Combos

Define specialized routing strategies by creating a combo JSON file and pointing the router to it via the OMNIROUTE_COMBO_PATH environment variable. The schema follows the definitions in open-sse/services/combo.ts, allowing you to customize the 15-factor scoring weights.

Remote Mode Deployment

Run OmniRoute on a VPS and connect via scoped tokens for team environments:

omniroute connect <host>

This mode uses token-based authentication as detailed in docs/guides/REMOTE-MODE.md.

Token Compression

Enable the Caveman or RTK compression engines to reduce token usage by up to 95%. Configuration details are available in docs/compression/COMPRESSION_ENGINES.md.

Summary

Setting up OmniRoute as a self-hosted AI gateway involves these key steps:

• Install the global npm package to get the CLI and embedded services.
• Run omniroute run to start the server on port 20128 with zero external configuration.
• Test the endpoint using model: "auto" to leverage the free-tier Auto-Combo routing.
• Configure additional providers via the dashboard or CLI, with keys encrypted in SQLite.
• Extend functionality using the MCP server at open-sse/mcp-server/server.ts or custom combo configurations.
• Scale using remote mode and compression engines for production deployments.

Frequently Asked Questions

Does OmniRoute require API keys to work immediately?

No. OmniRoute works out-of-the-box without any external API keys by routing requests through free-tier providers using the Auto-Combo engine. You only need to add paid provider keys if you require higher quotas or specific models not available on free tiers.

How does OmniRoute handle provider failures?

OmniRoute implements three independent resilience layers defined in docs/architecture/RESILIENCE_GUIDE.md: provider circuit breakers, connection cooldown periods, and model lockouts. These mechanisms automatically route around failing providers without manual intervention.

What is the Auto-Combo engine?

The Auto-Combo engine, implemented in open-sse/services/combo.ts, evaluates 19 routing strategies in real-time using a 15-factor scoring system. It dynamically selects the optimal provider based on latency, cost, availability, and request requirements, falling back to alternatives if quotas are exceeded.

Can I use OmniRoute with existing OpenAI SDKs?

Yes. OmniRoute exposes a fully OpenAI-compatible endpoint at /v1/chat/completions. You can point any standard OpenAI client to http://localhost:20128 and use it as a drop-in replacement, benefiting from unified access to 357 providers without code changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}