OmniRoute Project Structure: How Services Are Organized in the Open-SSE Architecture

OmniRoute organizes its request-handling pipeline into a modular service architecture where dedicated TypeScript modules handle combo routing, rate-limiting, credential gating, context compression, and fallback logic.

The diegosouzapw/OmniRoute repository implements a services-oriented architecture that decouples request handling into discrete, testable units. Instead of monolithic controllers, the project distributes concerns across the Open-SSE core, provider-specific helpers, and shared utility modules. This structure enables complex routing strategies—such as weighted round-robin and automatic failover—while maintaining clean separation between API endpoints, business logic, and external integrations.

High-Level Service Architecture

The OmniRoute project structure divides functionality into five distinct layers:

1. Next.js API Entry Points (src/app/api/v1/.../route.ts) – Thin validation wrappers that authenticate requests and delegate to the streaming engine.
2. Open-SSE Core (open-sse/handlers/…) – The central streaming layer that normalizes requests and invokes the appropriate executor.
3. Service Layer (open-sse/services/* and src/lib/services/*) – Pure TypeScript modules implementing rate-limiting, quota checks, fallback logic, and context handling.
4. Database Façade (src/lib/db/*) – CRUD helpers exposed through thin re-exports like src/lib/db/localDb.ts.
5. MCP/A2A Servers (open-sse/mcp-server/…, src/lib/a2a/…) – RPC-style services exposing diagnostics and combo metrics to external agents.

Core Combo Routing Service

At the heart of the OmniRoute project structure lies the combo routing service (open-sse/services/combo.ts). This module parses combo definitions (model combinations), expands wildcards, and applies routing strategies including priority, weighted distribution, and round-robin selection.

The handleComboChat function orchestrates the execution flow:

// open-sse/services/combo.ts (excerpt)
export async function handleComboChat(
  request: ChatRequest,
  options: HandleComboChatOptions,
) {
  const combo = resolveComboConfig(request);
  const targets = resolveComboTargets(combo);
  // Strategy selection, quota checks, and fallback handling
  const response = await executeRuntimeUnitCombo(targets, request);
  return buildPipelineResponse(response);
}

Key Collaborators in Combo Resolution

The combo service imports several specialized services to ensure reliable execution:

• rateLimitManager.ts – Tracks global per-provider rate limits to prevent quota exhaustion.
• providerCooldownTracker.ts – Records recent HTTP 429/503 responses to avoid hammering unstable providers.
• credentialGate.ts – Validates credential availability before attempting provider calls.
• contextManager.ts – Estimates token budgets and orchestrates context compression.
• modelFamilyFallback.ts – Selects alternative models within the same family when primaries fail.
• emergencyFallback.ts – Implements ultimate "any-provider" fallback when all configured targets are exhausted.

Rate-Limiting and Quota Services

Before any outbound HTTP call, the combo engine consults rate-limiting services to enforce traffic policies. The rateLimitSemaphore.ts module caps concurrent requests per provider, while quotaPreflight.ts retrieves current usage statistics (such as OpenAI token quotas) to reject requests early when limits are approaching.

Provider-Specific Service Implementations

Each supported provider maintains isolated service logic for authentication, transport, and rate-limiting within open-sse/services/:

• Anthropic: claudeTlsClient.ts and claudeTurnstileSolver.ts handle custom TLS requirements and challenge-solving.
• Gemini: geminiRateLimitTracker.ts and geminiThoughtSignatureStore.ts manage Google-specific quota and signature verification.
• OpenAI-Compatible: default.ts provides the base executor, while opencodeQuotaFetcher.ts retrieves usage metrics from compatible endpoints.

Context Compression and Token Management

Prompt optimization occurs before the combo engine executes. The context and compression services in open-sse/services/compression/* rewrite request bodies to fit token constraints.

The pipeline works as follows:

1. strategySelector.ts determines the compression mode (lite, caveman, or RTK) based on combo configuration.
2. caveman.ts executes heavyweight semantic condensation when aggressive compression is required.

Shared Infrastructure Services

Reusable utilities that support multiple request flows reside in src/lib/services/*:

• ringBuffer.ts – Implements a fixed-size ring buffer for sliding-window metrics calculations.
• reverseProxy.ts – Provides generic HTTP reverse-proxy functionality for internal tooling.

External Service Interfaces (MCP and A2A)

OmniRoute exposes its internal capabilities to external agents through standardized protocols. The MCP server (94 tools) and A2A server (JSON-RPC) load service catalogs from open-sse/mcp-server/ and src/lib/a2a/, respectively. These interfaces delegate to the same underlying services used by the core combo engine, enabling external tools to query combo metrics, invoke health checks, and manipulate routing configurations.

Request Flow Example

A chat completion request traverses the service layers as follows:

// Client call
await fetch('https://my-omniroute.local/api/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Explain microservices' }],
  }),
});

Server-side path:

1. src/app/api/v1/chat/completions/route.ts validates the request.
2. open-sse/handlers/chatCore.ts normalizes the payload.
3. open-sse/services/combo.ts resolves the combo configuration and executes the routing strategy.

MCP tools access the same services through a different entry point:

// MCP client pseudo-code
const { getComboMetrics } = await mcpClient.invoke(
  'get_combo_metrics', 
  { comboId: 'my-combo' }
);

This call routes through open-sse/mcp-server/tools/ and ultimately utilizes comboMetrics.ts.

Summary

• OmniRoute employs a layered service architecture separating API endpoints, core handlers, and business logic.
• The combo service (open-sse/services/combo.ts) coordinates model selection, fallback strategies, and quota validation.
• Provider-specific services isolate vendor logic for Anthropic, Gemini, and OpenAI-compatible endpoints.
• Context compression occurs pre-execution via strategySelector.ts and caveman.ts.
• MCP and A2A servers expose internal services to external agents using the same underlying modules.

Frequently Asked Questions

How does the combo routing service handle provider failures?

The handleComboChat function in open-sse/services/combo.ts implements a cascading fallback mechanism. It first attempts the primary model, then consults modelFamilyFallback.ts for alternatives within the same provider family, and finally invokes emergencyFallback.ts to select any available provider when configured targets exhaust.

What distinguishes the Open-SSE core from the service layer?

The Open-SSE core (open-sse/handlers/…) manages streaming infrastructure and request normalization, while the service layer (open-sse/services/*) contains pure business logic for routing, rate-limiting, and context management. Handlers import services, not vice versa, maintaining strict dependency direction.

How does OmniRoute integrate with external tools like MCP?

OmniRoute exposes its internal service catalog through the MCP server directory (open-sse/mcp-server/), which provides 94 tools that wrap functions like comboMetrics.ts. These endpoints allow external agents to query routing statistics and health checks without accessing the core API directly.

Where does context compression occur in the request lifecycle?

Context compression runs before the combo engine executes, within open-sse/services/compression/strategySelector.ts. The selector chooses between modes (lite, caveman, RTK) based on combo configuration, potentially invoking caveman.ts for semantic condensation to fit token budgets.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}