OmniRoute Main Modules Explained: 14 Core Components Powering 250+ AI Providers

OmniRoute's architecture consists of 14 specialized modules—API Layer, Open-SSE Core, Executors, Routing Engine, Database Layer, MCP Server, A2A Server, Guardrails, Compression Pipeline, Skills System, Electron Desktop Client, Tunnels & Proxies, Authz & Policy Engine, and Webhooks—each handling distinct concerns from request ingestion to provider execution.

OmniRoute is a unified AI proxy/router that consolidates 250+ AI providers behind a single flexible endpoint. Understanding its modular architecture is essential for anyone extending the system, debugging routing behavior, or deploying custom integrations. This guide breaks down each module, its source location, and how the pieces fit together.

---

API Layer: Public Endpoint Gateway

The API Layer exposes OpenAI-compatible REST endpoints using Next.js App Router. It handles CORS, request validation via Zod schemas, optional API-key authentication, and forwards validated requests to the core engine.

Key files in this module:

• src/app/api/v1/chat/completions/route.ts — Chat completion endpoint
• src/app/api/v1/embeddings/route.ts — Text embedding endpoint
• src/app/api/v1/images/generations/route.ts — Image generation endpoint
• src/app/api/v1/audio/speech/route.ts — Text-to-speech endpoint

Each route performs input validation before delegating to open-sse/handlers/chatCore.ts for execution.

---

Open-SSE Core: Streaming Execution Engine

The Open-SSE Core powers all LLM streaming operations. It performs request translation, executor selection, rate-limit enforcement, retry logic, and streams Server-Sent Events (SSE) back to clients.

Core components:

• open-sse/handlers/chatCore.ts — Main request handler and orchestrator
• open-sse/translator/index.ts — Request format translation between OpenAI schema and provider-native formats
• open-sse/executors/index.ts — Executor registry and dispatch

This module sits at the heart of OmniRoute's request lifecycle, bridging external APIs with internal routing decisions.

---

Executors: Provider-Specific Request Builders

Executors translate generic OpenAI-format requests into provider-specific HTTP calls. The modular executor system allows OmniRoute to handle provider quirks without polluting the core logic.

Executor	Purpose	Location
DefaultExecutor	Handles most OpenAI-compatible APIs	open-sse/executors/default.ts
CursorExecutor	Custom OAuth and headers for Cursor	open-sse/executors/cursor.ts
VertexExecutor	Google Cloud Vertex AI integration	open-sse/executors/vertex.ts
Cloudflare AI Executor	Workers AI with multipart payloads	open-sse/executors/cloudflare-ai.ts

Adding a new provider typically means implementing a new executor or extending DefaultExecutor with custom headers.

---

Routing / Combo Engine: Intelligent Request Distribution

The Combo Engine implements OmniRoute's auto-combo system with 17 routing strategies including weighted fallback, cost optimization, and context-aware routing. It determines which provider/model combination serves each request.

Key files:

• open-sse/services/combo.ts — Core routing logic and combo resolution
• src/shared/constants/routingStrategies.ts — Strategy definitions and priorities

This module enables features like "use GPT-4 for complex reasoning, fall back to GPT-3.5 for simple queries" or "route to cheapest available provider under latency constraints."

---

Database Layer: SQLite Persistence

OmniRoute uses SQLite via better-sqlite3 for all persistent state. The layer contains 95 domain-specific modules covering provider configs, combos, usage tracking, quota management, and compression statistics.

Critical files:

• src/lib/db/core.ts — Singleton DB instance creation, all access funnels through here
• src/lib/db/providers.ts — Provider configuration persistence
• src/lib/db/compressionCombos.ts — Compression settings per combo

The singleton pattern in core.ts ensures connection pooling efficiency across the application.

---

MCP Server: Multi-Tool Control Protocol

The MCP Server exposes 94 built-in tools for external integration, supporting stdio, SSE, and HTTP transports. Tools cover health monitoring, combo metrics, caching, compression, 1-proxy mode, memory management, skills, plugins, and third-party integrations (Notion, Obsidian vault).

Entry points:

• open-sse/mcp-server/server.ts — Server initialization and transport handling
• open-sse/mcp-server/tools/health.ts — Representative tool implementation

Invoke via command line:

omniroute --mcp list_combos

---

A2A Server: Agent-to-Agent Communication

The A2A Server provides JSON-RPC 2.0 endpoints for internal agents. It handles task lifecycle management, skill registration, and message routing between autonomous components.

Key files:

• src/lib/a2a/server.ts — JSON-RPC server and method routing
• src/lib/a2a/skills/smartRouting.ts — Skill-based routing intelligence

Example A2A invocation:

{
  "jsonrpc": "2.0",
  "method": "skill.execute",
  "params": {
    "name": "quotaManagement",
    "args": { "accountId": "12345" }
  },
  "id": 1
}

---

Guardrails: Runtime Safety and Privacy

Guardrails provide opt-in runtime filters for safety and privacy. Each guardrail can be disabled per-request via the x-omniroute-disabled-guardrails header.

Implemented protections:

• PII Masker — src/lib/guardrails/piiMasker.ts — Detects and redacts personally identifiable information
• Prompt Injection Guard — src/lib/guardrails/promptInjectionGuard.ts — Blocks known attack patterns

Guardrails run before request translation, ensuring sensitive data never reaches external providers.

---

Compression Pipeline: Token Usage Optimization

The Compression Pipeline proactively reduces prompt token count before provider transmission. Three engines are available: lite, caveman, and RTK (lossy semantic compression).

Configuration files:

• open-sse/services/compression/strategySelector.ts — Chooses engine based on combo settings
• open-sse/services/compression/engines/rtk/registry.ts — RTK compression registry

Enable per-combo via API:

PATCH /api/v1/combos/fast-combo HTTP/1.1
Content-Type: application/json

{
  "compressionMode": "lite"
}

---

Skills System: Extensible Plugin Architecture

The Skills System supports custom sandboxed plugins for business logic like quota management or routing recommendations. Skills are database-backed and invocable via MCP or A2A.

Core files:

• src/lib/skills/registry.ts — Skill discovery and metadata
• src/lib/skills/executor.ts — Sandboxed execution environment

Skills differ from MCP tools: they contain business logic and state, while MCP tools are thin wrappers around system functions.

---

Electron Desktop Client: Cross-Platform GUI

The Electron Desktop Client provides a graphical interface for local OmniRoute management. Built with Next.js App Router inside Electron, it enables combo configuration, log viewing, and routing engine interaction without command-line tools.

Source locations:

• electron/main.ts — Main process, window management, Node.js APIs
• electron/renderer/*.tsx — React components for the UI

---

Tunnels & Proxies: Network Connectivity Helpers

This module exposes OmniRoute through various tunneling solutions and provides debugging capabilities:

• src/lib/cloudflaredTunnel.ts — Cloudflare tunnel integration
• src/mitm/proxy.ts — MITM proxy for inspecting upstream traffic

Useful for development debugging or production deployment behind firewalls.

---

Authz & Policy Engine: Centralized Access Control

The Authz & Policy Engine classifies requests, evaluates policies, and enforces rules including API-key validation, quota checks, and compliance requirements.

Implementation:

• src/server/authz/classify.ts — Request classification into risk/permission categories
• src/server/authz/policyEngine.ts — Policy evaluation and decision enforcement

This engine runs early in the request lifecycle, before routing decisions are made.

---

Webhooks & Event System: Asynchronous Notifications

The Webhook Dispatcher sends secure, HMAC-signed callbacks with exponential backoff and automatic disabling after repeated failures.

Single entry point:

• src/lib/webhookDispatcher.ts

Configuration includes retry policies and circuit breaker thresholds.

---

Sample Integration Patterns

Basic Chat Completion Request

POST /api/v1/chat/completions HTTP/1.1
Host: localhost:3000
Content-Type: application/json
Authorization: Bearer <API_KEY>

{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "Explain OmniRoute modules" }],
  "temperature": 0.7
}

MCP Tool Discovery

# List all available combos through MCP

omniroute --mcp list_combos

---

Summary

• API Layer (src/app/api/v1/*/route.ts) — Validates and ingests requests
• Open-SSE Core (open-sse/handlers/chatCore.ts) — Orchestrates streaming execution
• Executors (open-sse/executors/*.ts) — Handle provider-specific protocol details
• Combo Engine (open-sse/services/combo.ts) — Routes across 250+ providers with 17 strategies
• Database Layer (src/lib/db/core.ts) — SQLite persistence via singleton pattern
• MCP Server (open-sse/mcp-server/server.ts) — Exposes 94 tools for external integration
• A2A Server (src/lib/a2a/server.ts) — JSON-RPC for agent communication
• Guardrails (src/lib/guardrails/*.ts) — Opt-in privacy and safety filters
• Compression Pipeline (open-sse/services/compression/) — Reduces token usage pre-transmission
• Skills System (src/lib/skills/) — Sandboxed plugin architecture
• Electron Client (electron/) — Cross-platform desktop GUI
• Tunnels & Proxies (src/lib/cloudflaredTunnel.ts, src/mitm/proxy.ts) — Network connectivity
• Authz Engine (src/server/authz/) — Centralized policy enforcement
• Webhooks (src/lib/webhookDispatcher.ts) — Reliable event notifications

---

Frequently Asked Questions

What is the difference between MCP tools and Skills in OmniRoute?

MCP tools are stateless system functions exposed through the Multi-tool Control Protocol—94 built-in tools for health checks, metrics, and integrations. Skills are stateful, sandboxed plugins with business logic that can maintain internal state and are invoked via A2A JSON-RPC. According to the OmniRoute source code, skills run in src/lib/skills/executor.ts while MCP tools execute directly from open-sse/mcp-server/tools/.

How does OmniRoute handle authentication across 250+ providers?

OmniRoute abstracts provider authentication through the Executors module. Each executor in open-sse/executors/*.ts handles provider-specific auth patterns—OAuth for Cursor, service accounts for Vertex AI, API keys for most others. The Authz & Policy Engine (src/server/authz/policyEngine.ts) validates OmniRoute's own API keys before forwarding to executors, creating a two-layer security model.

Can I disable specific guardrails for trusted workloads?

Yes. Guardrails are opt-in per-request via the x-omniroute-disabled-guardrails header. The guardrail implementations in src/lib/guardrails/piiMasker.ts and src/lib/guardrails/promptInjectionGuard.ts check this header before applying filters. This allows trusted internal applications to skip overhead while maintaining defaults for external traffic.

Where is request routing logic implemented in OmniRoute?

The Combo Engine in open-sse/services/combo.ts implements all routing decisions, supported by strategy definitions in src/shared/constants/routingStrategies.ts. This module evaluates 17 strategies including cost-optimization, latency constraints, weighted fallback, and context-aware routing to select the optimal provider/model combination for each request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}