How to Use OmniRoute in a Project: A Complete Integration Guide

OmniRoute is a self-hosted AI gateway that routes OpenAI-compatible requests to approximately 290 providers through a single local endpoint at http://localhost:20128/v1.

OmniRoute, maintained in the diegosouzapw/OmniRoute repository, provides a unified interface for AI model routing with built-in failover, token compression, and guardrails. By exposing a standard OpenAI API format, it allows any existing tool or codebase to access multiple AI backends without vendor lock-in or configuration changes.

Installation and Zero-Configuration Startup

OmniRoute distributes via npm and requires no API keys for initial testing. The built-in auto-combo feature automatically connects to free-tier providers like OpenCode Zen and Felo.

npm install -g omniroute
omniroute

The binary starts the gateway on 127.0.0.1:20128. The entry point for all chat completion requests is handled in src/app/api/v1/chat/completions/route.ts, which delegates to the combo engine in open-sse/services/combo.ts for provider selection.

Connecting OpenAI-Compatible Clients

Any tool supporting a custom --api-base or OPENAI_BASE_URL environment variable can connect immediately. This includes Claude Code, Cursor, Cline, and Codex.

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

The auto model parameter triggers the routing engine to evaluate provider health, quota availability, latency, and cost across all connected backends. The selection logic implements 19 built-in strategies defined in the combo service layer.

Routing Strategies and Model Selection

OmniRoute offers three primary methods for targeting AI providers: automatic selection, namespace prefixes, and custom combo definitions.

Using the Auto-Combo Engine

When you specify model: "auto", OmniRoute constructs a virtual combo from all available providers. The system scores candidates using a 12-factor algorithm (detailed in docs/routing/AUTO-COMBO.md) and routes to the optimal backend.

curl http://localhost:20128/v1/chat/completions \
  -d '{"model":"auto/cheap","messages":[{"role":"user","content":"Summarize this text"}]}'

The suffix /cheap modifies the strategy to prioritize cost optimization, resolved through open-sse/compression/strategySelector.ts.

Targeting Specific Providers

For deterministic routing, use namespace prefixes to address specific backends directly:

• oc/auto → OpenCode namespace
• felo/ → Felo namespace

curl http://localhost:20128/v1/chat/completions \
  -d '{"model":"oc/auto","messages":[{"role":"user","content":"Explain proxies"}]}'

Creating Custom Combos via CLI

Define persistent routing rules with weighted strategies or fallback chains using the CLI. Combos are stored in SQLite and expanded on every request.

omniroute combo create myCodingCombo \
  --steps 'openai/gpt-4o, anthropic/claude-3.5-sonnet, openrouter/auto' \
  --strategy weighted

The CLI command persists data via src/lib/db/combos.ts, and the executor factory in open-sse/executors/index.ts instantiates the appropriate provider handlers from open-sse/executors/default.ts.

Advanced Features and Configuration

OmniRoute includes enterprise-grade capabilities for optimization, security, and distributed deployment.

Token Compression

The gateway automatically applies lite compression (whitespace collapse, system prompt deduplication) to reduce token costs by 15–95%. For intensive workloads, enable the RTK engine:

omniroute compression set-engine rtk

Compression engines reside in open-sse/compression/engines/rtk/ and are selected dynamically by open-sse/compression/strategySelector.ts.

Guardrails and Security

The system provides prompt-injection detection and optional PII redaction through the guardrails layer in src/lib/guardrails/. These checks run before translation and execution.

Remote Mode Deployment

To run OmniRoute on a VPS or cloud instance while using local CLI tools:

omniroute connect 203.0.113.7
omniroute models list

The connect command generates a scoped token for secure remote access, maintaining access to combos, compression, and guardrails on the central instance.

MCP and A2A Integration

OmniRoute exposes 104 tools via the Message Control Protocol (MCP) for integration with agent frameworks. The MCP server implementation in open-sse/mcp-server/server.ts supports stdio, SSE, and HTTP transports.

import { spawn } from "child_process";

const mcp = spawn("omniroute", ["--mcp"]);
mcp.stdin.write(JSON.stringify({
  jsonrpc: "2.0",
  id: 1,
  method: "list_combos",
  params: {}
}) + "\n");

Additionally, the A2A (Agent-to-Agent) protocol endpoint at /.well-known/agent.json provides JSON-RPC 2.0 access for autonomous agents to manage routing quotas and memory.

Programmatic Integration Examples

Node.js Agent with Auto/Routing

import fetch from "node-fetch";

const response = await fetch("http://localhost:20128/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "auto/coding",
    messages: [{ role: "user", content: "Refactor this function for performance" }]
  })
});

const data = await response.json();
console.log(data.choices[0].message.content);

Creating Combos via REST API

For dynamic configuration without CLI access, use the REST endpoint defined in src/app/api/v1/combos/route.ts:

curl -X POST http://localhost:20128/v1/combos \
  -H "Content-Type: application/json" \
  -d '{
        "id": "fallbackChain",
        "steps": [
          {"provider":"anthropic","model":"claude-3.5-sonnet"},
          {"provider":"openai","model":"gpt-4o-mini"}
        ],
        "strategy":"priority"
      }'

Request/response translation between provider formats (OpenAI, Anthropic, Gemini) is handled by open-sse/translator/index.ts, ensuring normalized input/output regardless of the backend.

Interactive CLI Usage

For debugging or manual testing, the built-in TUI provides direct access to all gateway features:

omniroute chat        # Interactive chat with /model, /combo commands

omniroute doctor      # Health check for ports, dependencies, providers

omniroute setup       # Wizard for registering paid provider API keys

Summary

• OmniRoute acts as a self-hosted gateway unifying ~290 AI providers under a single OpenAI-compatible endpoint at localhost:20128/v1.
• Zero-config installation via npm install -g omniroute immediately enables routing through free-tier providers using the auto model parameter.
• Three-layer architecture: Routing/combo engine (open-sse/services/combo.ts), provider executors (open-sse/executors/index.ts), and compression/guardrails layers handle request lifecycle.
• Flexible targeting allows automatic load balancing, namespace-prefixed specific providers (oc/, felo/), or custom SQLite-backed combos with 19 routing strategies.
• Enterprise features include RTK token compression (15–95% savings), prompt-injection guardrails, remote server mode, and MCP/A2A protocol support for agentic workflows.

Frequently Asked Questions

Do I need API keys to start using OmniRoute?

No. OmniRoute includes built-in connections to free providers like OpenCode Zen and Felo, allowing you to run omniroute and execute requests immediately without configuration. API keys are only required when adding paid providers via omniroute setup or the combo configuration interface.

How does OmniRoute handle provider-specific request formats?

The gateway uses a translator abstraction layer implemented in open-sse/translator/index.ts that normalizes all incoming OpenAI-format requests into provider-native formats (Anthropic Message API, Gemini, etc.) and transforms responses back to OpenAI format. This occurs before the executor in open-sse/executors/default.ts dispatches the HTTP call.

Can I use OmniRoute with existing OpenAI SDK codebases?

Yes. Set the baseURL (or OPENAI_BASE_URL environment variable) to http://localhost:20128/v1. The SDK will communicate with OmniRoute exactly as it would with OpenAI's official endpoint, while OmniRoute's routing engine in src/app/api/v1/chat/completions/route.ts handles provider selection transparently.

What is the difference between auto and auto/cheap model selections?

auto triggers the standard combo engine that balances quality, latency, and cost. auto/cheap applies a specific routing strategy that prioritizes cost optimization, filtering for the lowest-priced available providers first. Both strategies are evaluated in open-sse/services/combo.ts based on real-time quota and health metrics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}