How to Set Up OmniRoute as a Local AI Gateway: Complete Self-Hosted Setup Guide

OmniRoute is a self-hosted AI gateway that exposes OpenAI-compatible endpoints on localhost:20128 with zero-configuration setup, automatically routing LLM requests across 351 providers through a Next.js 16 streaming architecture.

OmniRoute (diegosouzapw/OmniRoute) operates as a local AI gateway that aggregates dozens of LLM providers behind a single unified interface. Unlike cloud-based solutions, this open-source tool runs entirely on your infrastructure, offering complete control over request routing, provider selection, and data privacy. The system requires no API keys out of the box, utilizing the free "OpenCode" provider to handle initial requests immediately after installation.

Prerequisites and Installation

You can install OmniRoute via npm or Docker. The npm package provides a global CLI command, while the Docker image offers containerized deployment.

NPM Installation:

npm install -g omniroute

Docker Installation:

docker run -p 20128:20128 diegosouzapw/omniroute

The gateway requires Node.js 16+ for native execution and exposes port 20128 by default.

Architecture Overview

OmniRoute's architecture centers on a Next.js 16 application that processes HTTP requests through a modular pipeline. The system separates concerns between API routing, request handling, provider execution, and response translation.

Request Flow Pipeline

When you send a request to http://localhost:20128/v1/chat/completions, OmniRoute processes it through five distinct stages:

1. API Entry Point – Routes in src/app/api/v1/* validate payloads using Zod schemas before forwarding to handlers.
2. Core Handler – open-sse/handlers/chatCore.ts (via handleChatCore) orchestrates the request pipeline and applies policy checks.
3. Combo Routing – open-sse/services/combo.ts implements 19 routing strategies (including auto-combo) to select optimal providers.
4. Provider Execution – Executor modules in open-sse/executors/ handle HTTP communication with upstream LLM services.
5. Response Translation – Translators in open-sse/translator/ normalize provider-specific payloads into the OpenAI schema before streaming via SSE.

Key Source Files

Understanding these critical files helps with customization and debugging:

• bin/omniroute.mjs – Server entry point that initializes the Next.js application and loads gateway configurations.
• src/app/api/v1/chat/completions/route.ts – Receives OpenAI-compatible POST requests and manages the initial request validation.
• open-sse/handlers/chatCore.ts – Contains handleChatCore function that manages the complete request lifecycle.
• open-sse/services/combo.ts – Implements the combo routing engine supporting strategies like auto-combo and fill-first.
• open-sse/executors/* – Directory containing provider-specific HTTP logic for each supported LLM service.
• open-sse/translator/* – Modules that transform diverse provider response formats into standard OpenAI-compatible streams.
• docs/reference/PROVIDER_REFERENCE.md – Catalog documenting all 351 available providers and their free-tier availability.
• src/app/(dashboard)/** – Web interface components for monitoring quotas, provider status, and routing policies.

Starting the Gateway

Launching OmniRoute requires a single command after installation. The server entry point in bin/omniroute.mjs automatically configures the environment and binds to the default port.

omniroute

The service starts on http://localhost:20128 and immediately begins accepting requests. The auto-combo strategy activates automatically, including the free "OpenCode" provider so your first request succeeds without API keys.

To verify the server is running:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

Making Requests to Local Endpoints

OmniRoute accepts standard OpenAI SDK calls by changing the base URL to your local gateway. All compatible clients can point at http://localhost:20128/v1 and issue normal /chat/completions calls.

cURL Example:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Explain recursion"}]}'

Python OpenAI SDK Example:

import openai

openai.api_base = "http://localhost:20128/v1"
openai.api_key = "unused"  # Required by SDK but ignored by gateway

response = openai.ChatCompletion.create(
    model="auto",
    messages=[{"role": "user", "content": "Explain the Fibonacci sequence"}]
)
print(response.choices[0].message.content)

The model: "auto" parameter triggers the combo router in open-sse/services/combo.ts to intelligently select between available providers based on cost, latency, and availability.

Configuring Providers and Routing

While OmniRoute works zero-config using the built-in OpenCode provider, you can customize the 351-provider catalog documented in docs/reference/PROVIDER_REFERENCE.md.

Accessing the Dashboard:

Navigate to http://localhost:20128/dashboard/providers to view the web UI implemented in src/app/(dashboard)/. This interface allows you to enable, disable, or prioritize specific providers without editing configuration files.

Adding Custom Providers:

For enterprise or private endpoints, add providers programmatically:

curl -X POST http://localhost:20128/api/v1/providers \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-custom-provider",
    "apiKey": "sk-your-key-here",
    "baseUrl": "https://api.custom-provider.com/v1"
  }'

Advanced Routing Strategies

The combo service in open-sse/services/combo.ts implements 19 distinct routing strategies beyond the default auto-combo. These include fill-first (prioritizing cheapest available), round-robin (load balancing), and failover (cascading through providers on error).

To specify a strategy, include it in your request parameters or configure it as the default in the dashboard. The handler in open-sse/handlers/chatCore.ts evaluates these policies before executing provider calls.

Summary

• OmniRoute provides a zero-config local AI gateway at localhost:20128 with OpenAI-compatible endpoints.
• The architecture uses Next.js 16 for API routing (src/app/api/v1/*) and open-sse modules for streaming execution.
• Auto-combo routing automatically includes the free OpenCode provider, enabling immediate testing without API keys.
• The system supports 351 providers cataloged in docs/reference/PROVIDER_REFERENCE.md with configurable strategies via open-sse/services/combo.ts.
• Both npm (npm i -g omniroute) and Docker deployment options require only port 20128 exposure to begin serving requests.

Frequently Asked Questions

What port does OmniRoute use by default?

OmniRoute binds to port 20128 by default. The server entry point in bin/omniroute.mjs initializes the Next.js application on this port, making the gateway accessible at http://localhost:20128. You can map this to any host port when using Docker (-p 20128:20128).

Does OmniRoute require API keys to start?

No. OmniRoute operates zero-config on first launch by automatically enabling the free "OpenCode" provider through the auto-combo strategy implemented in open-sse/services/combo.ts. This allows immediate testing of the /v1/chat/completions endpoint without configuring credentials.

How does OmniRoute handle different LLM provider formats?

OmniRoute normalizes responses through a dedicated translation layer located in open-sse/translator/. After executor modules in open-sse/executors/ fetch responses from upstream providers, translators convert proprietary schemas into the standard OpenAI-compatible format before streaming results to clients via Server-Sent Events (SSE).

Can I use standard OpenAI client libraries with OmniRoute?

Yes. Any OpenAI-compatible client library works by changing the api_base or base_url parameter to http://localhost:20128/v1. The API routes in src/app/api/v1/chat/completions/route.ts accept standard request payloads, allowing drop-in replacement of OpenAI's official endpoint with your local gateway.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}