What Is the OmniRoute Combo Routing Engine? A Complete Technical Guide

The OmniRoute Combo Routing Engine is the core request-dispatcher in the diegosouzapw/OmniRoute open-source proxy that intelligently routes LLM API calls across multiple provider-model pairs using 19 configurable strategies, built-in resilience patterns, and optional multi-model fusion.

When a client calls POST /v1/chat/completions and includes a combo name in the request payload, the engine orchestrates a three-stage pipeline to determine which providers should receive the request, how many parallel calls to execute, and how to combine the responses into a single coherent output.

Three-Stage Routing Architecture

The engine processes every request through distinct phases implemented in the open-sse/services/combo/ directory, transforming a combo definition into a final LLM response.

Stage 1: Combo Setup and Strategy Resolution

In comboSetup.ts, the phaseComboSetup(ctx) function initializes the routing context by extracting the combo name from the HTTP request and normalizing the routing strategy via normalizeRoutingStrategy. This stage applies user-defined context-relay and universal-handoff options that allow request rewriting before dispatch, loads resilience settings including circuit-breakers and cooldowns, and optionally pins the model to the last-used one for the session using server-side context caching.

Stage 2: Target Resolution and Filtering

The resolveComboTargets(setup, ctx) function in targetResolution.ts transforms the combo definition—stored in the combos table and accessed via src/lib/db/combos.ts—into an ordered list of candidate targets. This phase filters out unavailable accounts by checking circuit-breaker states and connection-level cooldowns defined in src/shared/utils/circuitBreaker.ts, applies quota and capacity heuristics to determine provider headroom, and validates eligibility using comboPredicates.ts to enforce cost limits and feature flags.

Stage 3: Strategy Dispatch and Execution

strategyDispatch.ts contains the dispatchCombo(setup, targets, ctx) implementation that executes the chosen strategy. For single-target strategies like priority, round-robin, or least-used, the engine iterates sequentially through the target list until a successful response is obtained. For multi-target strategies like fusion, the engine fans out requests in parallel and aggregates results. Errors from individual providers are captured, classified, and fed back into the resilience subsystem to update circuit-breaker states and model lockouts.

Routing Strategies and Selection Logic

OmniRoute implements 19 built-in strategies, each optimizing for distinct operational constraints ranging from static prioritization to dynamic adaptive selection.

Single-Target Selection Strategies

These strategies identify one provider-model pair per request and are implemented in the target sorters:

• Priority: Routes to the first available target in the statically ordered list defined in the combo configuration.
• Round-robin: Distributes load cyclically across all available targets in the combo.
• Least-used: Selects the target with the lowest recent utilization metrics tracked in runtimeUnits.ts.
• Cost-optimized: Prioritizes models with the lowest token pricing based on current rate cards.
• Headroom: Chooses providers with the greatest remaining capacity quota to prevent rate limit exhaustion.

Multi-Model Fusion Strategy

The fusion strategy, implemented in fusionPanel.ts, sends the request to a panel of models in parallel. A designated judge model—configured via the judgeModel field in the combo definition—synthesizes the panel outputs into a single final answer. This enables ensemble reasoning across heterogeneous LLM providers such as OpenAI, Anthropic, and Google Gemini simultaneously.

Auto-Adaptive Strategy

The auto strategy in autoStrategy.ts computes dynamic scores for each target using real-time metrics including latency, success rate, and token usage tracked in runtimeUnits.ts and runtimeUnitCapacity.ts. Targets are ordered by this computed score, allowing the engine to adapt routing decisions to changing provider health without requiring code deployments or configuration changes.

Resilience and Error Handling

The engine integrates multiple resilience layers to prevent cascade failures and provide transparent error reporting.

Before dispatch, each target undergoes validation against provider-level circuit breakers and connection-level cooldowns. When individual targets fail, errors are captured in strategyDispatch.ts and used to update resilience states. If all targets in a combo fail, comboErrorAggregation.ts collates the individual provider errors into a structured response that details the specific failure reason for each attempted provider.

Practical Implementation Examples

Creating a Combo Definition

Define routing groups using the database API in src/lib/db/combos.ts:

import { createCombo } from "@/src/lib/db/combos.ts";

await createCombo({
  name: "fast-cheap-combo",
  description: "Prioritises cheap, fast models",
  strategy: "priority",
  targets: [
    { provider: "openai", model: "gpt-3.5-turbo", accountId: "key-1" },
    { provider: "anthropic", model: "claude-1.2", accountId: "key-2" },
  ],
});

Invoking the Combo via API

Pass the combo name in the combo field when calling the chat completions endpoint:

curl -X POST http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "omni",
        "messages": [{"role":"user","content":"Hello"}],
        "combo": "fast-cheap-combo",
        "stream": false
      }'

Configuring Multi-Model Fusion

Set up ensemble reasoning with a judge model to synthesize parallel responses:

await createCombo({
  name: "fusion-review",
  strategy: "fusion",
  judgeModel: { provider: "openai", model: "gpt-4" },
  targets: [
    { provider: "openai", model: "gpt-3.5-turbo", accountId: "key-1" },
    { provider: "anthropic", model: "claude-1.2", accountId: "key-2" },
    { provider: "google", model: "gemini-1.0", accountId: "key-3" },
  ],
});

Debugging Target Selection

Enable verbose logging to inspect strategy selection and target ordering by setting the environment variable OMNIROUTE_DEBUG_COMBO=1 or adding debug: true to the request payload. The engine logs decisions via the ComboContext logger in comboSetup.ts, outputting lines such as Chosen strategy=priority, targets=openai/gpt-3.5-turbo,anthropic/claude-1.2.

Summary

• The OmniRoute Combo Routing Engine processes requests through three phases—setup, target resolution, and dispatch—implemented in comboSetup.ts, targetResolution.ts, and strategyDispatch.ts respectively.
• 19 built-in strategies range from static priority routing to dynamic auto-selection and multi-model fusion, accommodating diverse latency, cost, and reliability requirements.
• Resilience patterns including circuit breakers, cooldowns, and error aggregation prevent provider failures from affecting service availability.
• Fusion capabilities enable parallel execution across multiple models with judge-based synthesis for improved response quality.
• Configuration occurs via the combos database table and runtime request parameters, allowing flexible routing without code changes.

Frequently Asked Questions

How does the OmniRoute Combo Routing Engine handle provider failures?

When a provider fails, the engine captures the error in strategyDispatch.ts, updates the circuit-breaker state in src/shared/utils/circuitBreaker.ts, and immediately retries with the next available target in the ordered list. If all targets fail, comboErrorAggregation.ts returns a comprehensive error report to the client detailing each provider's specific failure reason.

What is the difference between the priority and auto strategies?

Priority follows a static ordered list defined in the combo configuration and routes to the first healthy provider, while auto dynamically recalculates target scores every request based on real-time metrics including latency and success rates stored in runtimeUnits.ts, automatically favoring the healthiest provider without manual configuration updates.

Can I combine responses from multiple models into a single output?

Yes, using the fusion strategy implemented in fusionPanel.ts. Configure strategy: "fusion" in your combo definition and specify a judgeModel property. The engine will fan out requests to all panel targets in parallel, then use the judge model to synthesize a unified response from the individual model outputs.

Where are combo definitions stored and how are they accessed?

Combo definitions persist in the PostgreSQL combos table managed by src/lib/db/combos.ts. During the target resolution phase in targetResolution.ts, the engine reads these definitions to retrieve provider lists, strategy overrides, and account credentials, applying them at runtime to determine routing decisions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}