Main Modules in OmniRoute: A Deep Dive into the Unified AI Router Architecture

OmniRoute organizes its functionality into 14 distinct modules—including an API Layer, Open‑SSE Core, Executors, Routing Engine, Database Layer, MCP Server, A2A Server, Guardrails, and Compression Pipeline—that collectively manage 250+ AI providers behind a single endpoint.

OmniRoute is a unified AI proxy and router that consolidates over 250 AI providers behind a single, flexible API. Understanding the main modules in OmniRoute is essential for developers who want to extend the system, debug routing behavior, or integrate custom providers. The architecture follows a modular design where each component handles a specific concern—from request validation and streaming execution to persistence and safety guardrails.

API Layer: Next.js App Router Endpoints

The API Layer exposes standard OpenAI-compatible endpoints for chat completions, embeddings, image generation, and audio processing. Built on the Next.js App Router, this module handles CORS configuration, request validation via Zod schemas, optional API‑key authentication, and forwards sanitized requests to the core engine.

Key source files include:

• src/app/api/v1/chat/completions/route.ts – Handles chat completion requests
• src/app/api/v1/embeddings/route.ts – Manages text embedding requests
• src/app/api/v1/images/generations/route.ts – Processes image generation calls
• src/app/api/v1/audio/speech/route.ts – Routes text‑to‑speech requests

Open‑SSE Core: The Streaming Engine

The Open‑SSE Core module powers all LLM streaming calls. It performs request translation, selects the appropriate executor, applies rate‑limit and retry logic, and streams Server‑Sent Events (SSE) back to the client.

Critical components reside in:

• open-sse/handlers/chatCore.ts – Central handler for chat request orchestration
• open-sse/translator/index.ts – Request format translation layer
• open-sse/executors/index.ts – Executor selection and dispatch

Executors: Provider‑Specific Request Builders

Executors translate generic OmniRoute requests into provider‑specific formats. The generic DefaultExecutor covers most OpenAI‑compatible APIs, while specialized executors handle provider quirks such as OAuth flows, custom headers, or multipart payloads.

Notable implementations include:

• open-sse/executors/default.ts – Standard OpenAI‑compatible execution
• open-sse/executors/cursor.ts – Cursor‑specific request handling
• open-sse/executors/vertex.ts – Google Vertex AI integration

Routing and Combo Engine

The Routing / Combo Engine implements the auto‑combo system with 17 routing strategies, combo definitions, weighted fallback logic, cost‑optimization, and context‑aware routing. This module determines which provider or model should serve each request based on availability, cost, and performance criteria.

Core files:

• open-sse/services/combo.ts – Combo resolution and routing logic
• src/shared/constants/routingStrategies.ts – Strategy definitions and weights

Database Layer: SQLite Persistence

The Database Layer uses better‑sqlite3 for persistent storage, managing 95 domain‑specific modules that store provider configurations, combo definitions, usage statistics, quota tracking, and compression metrics. All database access flows through a singleton instance created in src/lib/db/core.ts.

Key database modules:

• src/lib/db/core.ts – Singleton database instance and connection management
• src/lib/db/providers.ts – Provider configuration persistence
• src/lib/db/compressionCombos.ts – Compression settings per combo

MCP Server: Multi‑Tool Protocol

The MCP Server exposes 94 built‑in tools via the Meta‑Control‑Protocol, supporting stdio, SSE, and HTTP transports. Tools cover health checks, combo metrics, cache management, compression controls, and integrations with Notion and Obsidian vaults.

Primary implementation:

• open-sse/mcp-server/server.ts – Main MCP server entry point
• open-sse/mcp-server/tools/health.ts – Representative health check tool

A2A Server: Agent‑to‑Agent Communication

The A2A Server provides a JSON‑RPC 2.0 interface for internal agents and skills. It handles task lifecycle management, skill registration, and message routing between autonomous components.

Source locations:

• src/lib/a2a/server.ts – JSON‑RPC server implementation
• src/lib/a2a/skills/smartRouting.ts – Smart routing skill example

Guardrails: Runtime Safety and Privacy

Guardrails enforce runtime safety through opt‑in filters for PII redaction, prompt‑injection detection, and vision‑bridge policy enforcement. Each guardrail can be disabled per‑request via the x‑omniroute‑disabled‑guardrails header.

Implementation files:

• src/lib/guardrails/piiMasker.ts – Personal information detection and masking
• src/lib/guardrails/promptInjectionGuard.ts – Prompt injection attack prevention

Compression Pipeline

The Compression Pipeline proactively reduces token usage through three engines: lite, caveman, and RTK. Configurable per‑combo, this module compresses prompts before they reach the upstream provider, significantly reducing costs.

Core components:

• open-sse/services/compression/strategySelector.ts – Engine selection logic
• open-sse/services/compression/engines/rtk/registry.ts – RTK compression engine registry

Skills System: Extensible Plugin Architecture

The Skills System provides a sandboxed, database‑backed plugin architecture for custom business logic. Skills can be invoked via MCP or A2A interfaces and run in isolated execution contexts.

Key files:

• src/lib/skills/registry.ts – Skill discovery and registration
• src/lib/skills/executor.ts – Sandboxed skill execution environment

Electron Desktop Client

The Electron Desktop Client delivers a cross‑platform graphical interface built with Next.js and Electron. It provides local management of combos, real‑time log viewing, and direct interaction with the routing engine without requiring command‑line interaction.

Entry points:

• electron/main.ts – Electron main process
• electron/renderer/*.tsx – Renderer components

Supporting Infrastructure

Beyond the core modules, OmniRoute includes several specialized infrastructure components:

Tunnels & Proxies – Exposes OmniRoute through Cloudflare, Ngrok, or Tailscale tunnels, plus an MITM proxy for debugging upstream traffic (src/lib/cloudflaredTunnel.ts, src/mitm/proxy.ts).

Authz & Policy Engine – Centralizes request classification, policy evaluation, and enforcement including API‑key validation and quota checks (src/server/authz/classify.ts, src/server/authz/policyEngine.ts).

Webhooks & Event System – Dispatches secure, HMAC‑signed webhooks with exponential back‑off and auto‑disable logic after repeated failures (src/lib/webhookDispatcher.ts).

Utilities & Types – Shared TypeScript definitions, Zod schemas, logger helpers, and diagnostics (src/types/*, open-sse/utils/estimateSize.ts).

Practical Usage Examples

The following examples demonstrate how clients interact with OmniRoute’s public APIs and internal tooling.

Sending a Chat Completion Request

POST /api/v1/chat/completions HTTP/1.1
Host: localhost:3000
Content-Type: application/json
Authorization: Bearer <API_KEY>

{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "What is the capital of France?" }],
  "temperature": 0.7
}

This request hits src/app/api/v1/chat/completions/route.ts, undergoes Zod validation, API‑key verification, and delegates to open-sse/handlers/chatCore.ts for combo routing and SSE streaming.

Querying Provider Combos via MCP

omniroute --mcp list_combos

The MCP server (open-sse/mcp-server/server.ts) authenticates the client scope and returns a JSON list of all defined routing combos.

Invoking a Custom Skill via A2A

{
  "jsonrpc": "2.0",
  "method": "skill.execute",
  "params": {
    "name": "quotaManagement",
    "args": { "accountId": "12345" }
  },
  "id": 1
}

The A2A server (src/lib/a2a/server.ts) resolves the quotaManagement skill from src/lib/skills/registry.ts and executes it in the sandboxed environment.

Enabling Prompt Compression for a Combo

PATCH /api/v1/combos/fast-combo HTTP/1.1
Content-Type: application/json

{
  "compressionMode": "lite"
}

This updates the combo configuration in src/lib/db/compressionCombos.ts; subsequent requests trigger the lite compression pipeline (open-sse/services/compression/lite.ts).

Summary

The main modules in OmniRoute form a cohesive, extensible architecture for managing AI provider diversity at scale:

• API Layer – Next.js endpoints for OpenAI‑compatible chat, embeddings, images, and audio
• Open‑SSE Core – Central streaming engine handling translation, retries, and SSE output
• Executors – Provider‑specific request builders for 250+ backends
• Routing Engine – 17‑strategy combo system with cost optimization and fallback logic
• Database Layer – SQLite persistence via better‑sqlite3 with 95 domain modules
• MCP Server – 94 built‑in tools exposed via Meta‑Control‑Protocol
• A2A Server – JSON‑RPC 2.0 interface for agent skills and task management
• Guardrails – Opt‑in PII masking and prompt injection protection
• Compression Pipeline – Proactive token reduction via lite, caveman, and RTK engines
• Skills System – Sandboxed, database‑backed plugin architecture
• Desktop Client – Electron‑based GUI for local management
• Infrastructure – Tunnels, Authz policy engine, and webhook dispatchers

Frequently Asked Questions

What is the Open‑SSE Core responsible for in OmniRoute?

The Open‑SSE Core is the central streaming engine that handles all LLM requests. Located in open-sse/handlers/chatCore.ts, it translates incoming requests, selects the appropriate executor from open-sse/executors/index.ts, applies rate‑limiting and retry logic, and streams Server‑Sent Events back to clients.

How does OmniRoute handle routing between different AI providers?

OmniRoute uses the Routing / Combo Engine implemented in open-sse/services/combo.ts and src/shared/constants/routingStrategies.ts. This system evaluates 17 distinct routing strategies—including cost optimization, latency‑based selection, and weighted fallback—to determine the optimal provider for each request.

Can I disable safety guardrails for specific requests?

Yes. OmniRoute’s Guardrails module supports per‑request disabling via the x‑omniroute‑disabled‑guardrails header. Individual guardrails for PII redaction (src/lib/guardrails/piiMasker.ts) and prompt injection detection (src/lib/guardrails/promptInjectionGuard.ts) are opt‑in and can be selectively disabled when necessary.

What database does OmniRoute use for persistence?

OmniRoute uses SQLite via the better‑sqlite3 driver. The database layer in src/lib/db/core.ts creates a singleton connection instance, while specialized modules like src/lib/db/providers.ts and src/lib/db/compressionCombos.ts handle specific domain persistence for 95 different data types including provider configs, usage stats, and quota tracking.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}