How OmniRoute's Combo Routing Engine Manages 17 Routing Strategies: From Priority to Weighted Round-Robin

OmniRoute's combo routing engine processes requests through a strategy-agnostic pipeline that resolves target models, applies strategy-specific ordering via the applyStrategyOrdering pure function, and executes targets sequentially with health checks and sticky session support.

The diegosouzapw/OmniRoute repository implements a sophisticated combo routing engine capable of distributing AI model requests across 17 distinct routing strategies. At its core, the system separates strategy selection from target resolution, enabling flexible load balancing across multiple providers while maintaining clean separation between ordering logic and execution.

Core Architecture of the Combo Routing Engine

The combo routing engine operates through a four-phase pipeline defined in open-sse/services/combo.ts:

1. Strategy Extraction: The handleComboChat function extracts the strategy field from the combo definition at line 773. Special strategies like "fusion" and "pipeline" branch to dedicated handlers, while standard strategies proceed through the generic pipeline.

2. Target Resolution: All model entries—including provider wildcards and fingerprint expansions—resolve into a flat array of ResolvedComboTarget objects via resolveComboTargets near line 6575.

3. Strategy Ordering: For every non-auto strategy, the pure function applyStrategyOrdering (in open-sse/services/combo/applyStrategyOrdering.ts) re-orders the target list according to the selected algorithm without side effects.

4. Sequential Execution: The ordered list traverses through executeRuntimeUnitCombo (lines 858-901), where each target attempts execution with per-target timeouts, progressing to the next target on failure.

Routing Strategy Implementations

OmniRoute supports 17 distinct strategies, each implemented as a specific ordering algorithm within applyStrategyOrdering.ts or through specialized logic in handleComboChat.

Sequential Strategies

Priority (the default) maintains the original list order. Before execution, a pre-screen check (lines 12004-12011 in combo.ts) verifies latency and health for all targets, allowing the engine to skip unhealthy targets without waiting for timeouts.

Fill-First preserves priority order but optimizes for quota completion, logging the strategy selection without reordering (lines 120-124 in applyStrategyOrdering.ts).

Distribution Strategies

Weighted routing distributes traffic proportionally using sticky session support. The engine generates a sticky key via getStickyWeightedExecutionKey (lines 858-874 in combo.ts); if the current target remains eligible, it moves to the front and the strategy falls back to priority. Otherwise, fisherYatesShuffle randomizes the remaining targets before weighted selection. Success triggers recordStickyWeightedSuccess to update counters (lines 938-940).

Round-Robin rotates through targets using a per-combo sticky limit stored in rrStickyTargets. The getStickyRoundRobinStartIndex function calculates the rotation index (lines 888-904), while recordStickyRoundRobinSuccess updates state after successful requests (lines 945-954).

Random applies fisherYatesShuffle to the entire target list on every request (lines 117-119).

Strict-Random implements deterministic deck-based selection. Using a persistent deck key (combo:<name>), getNextFromDeck selects the primary target while shuffling remaining fallbacks (lines 96-112).

Optimization Strategies

Least-Used favors targets with the lowest invocation count by calling sortTargetsByUsage from targetSorters.ts (lines 128-131).

Cost-Optimized ranks targets by estimated cost per token using sortTargetsByCost, with optional manifest routing hints (lines 131-172).

Context-Optimized prioritizes models with the largest context windows via sortTargetsByContextSize (lines 196-199).

Headroom selects targets with the most available capacity using orderTargetsByHeadroom (lines 199-210).

Quota-Aware Strategies

Reset-Aware and Reset-Window prefer targets closest to quota replenishment through orderTargetsByResetAwareQuota and orderTargetsByResetWindow (lines 172-196).

Quota-Share implements a distributed round-robin (DRR) plus power-of-two-choices (P2C) algorithm via selectQuotaShareTarget in quotaShareStrategy.ts. This respects per-connection concurrency limits resolved through resolveMaxConcurrentByConnection (lines 210-223).

Dynamic Strategies

Auto delegates strategy selection to resolveAutoStrategyOrder (starting at line 1087), which builds candidate objects, scores them based on live metrics (latency, success rate, cost), and returns an ordered list.

Configuration Examples

Weighted Strategy with Sticky Sessions

{
  "name": "my-weighted-combo",
  "strategy": "weighted",
  "config": {
    "stickyWeightedLimit": 3
  },
  "models": [
    "openai/gpt-4o-mini",
    "anthropic/claude-3.5-sonnet",
    "groq/llama-3.1-70b"
  ]
}

This configuration pins requests to a single target for up to three consecutive calls before re-evaluating weights.

Round-Robin with Sticky Limits

{
  "name": "rr-combo",
  "strategy": "round-robin",
  "config": {
    "stickyRoundRobinLimit": 5
  },
  "models": [
    "openai/gpt-4o",
    "openai/gpt-4o-mini",
    "anthropic/claude-3-opus"
  ]
}

The engine rotates through targets after every five requests, balancing load while maintaining temporary session affinity.

Priority with Pre-Screening

{
  "name": "priority-combo",
  "strategy": "priority",
  "models": [
    "openai/gpt-4o-mini",
    "anthropic/claude-3-sonnet"
  ]
}

Targets execute in list order, with the pre-screen step potentially skipping unhealthy providers before attempting connections.

Key Source Files

The combo routing engine spans several modules:

• open-sse/services/combo.ts: Contains handleComboChat (the main dispatcher), resolveComboTargets, and executeRuntimeUnitCombo (the sequential execution loop).
• open-sse/services/combo/applyStrategyOrdering.ts: Pure function implementing all non-auto strategy ordering algorithms.
• open-sse/services/combo/quotaShareStrategy.ts: DRR and P2C logic for quota-share routing.
• open-sse/services/combo/resolveAutoStrategy.ts: Dynamic strategy selection and candidate scoring.
• src/shared/utils/shuffleDeck.ts: Deck-based deterministic randomization for strict-random strategies.

Summary

• OmniRoute's combo routing engine separates strategy selection from target execution, enabling 17 distinct routing algorithms through a unified pipeline.
• The applyStrategyOrdering pure function handles strategy-specific reordering, while handleComboChat manages sticky sessions and pre-screening.
• Weighted and round-robin strategies support sticky limits to balance load distribution with session affinity.
• Quota-share implements sophisticated DRR+P2C algorithms respecting concurrency limits and bucket quotas.
• All strategies integrate with the same sequential execution loop, ensuring consistent timeout handling and fallback behavior across the entire system.

Frequently Asked Questions

How does OmniRoute handle unhealthy targets in priority routing?

The engine runs a pre-screen check before executing priority combos, verifying latency and health for all targets defined at lines 12004-12011 in combo.ts. This allows the system to skip failing early targets without waiting for full request timeouts, maintaining low latency even when primary providers are degraded.

What is the difference between random and strict-random strategies?

Random shuffles the target list using fisherYatesShuffle on every request (lines 117-119), providing non-deterministic distribution. Strict-random uses a persistent deck mechanism via getNextFromDeck (lines 96-112) to ensure deterministic selection from a pre-shuffled order, while still randomizing fallback targets for resilience.

How does the weighted strategy maintain session affinity?

The weighted strategy implements sticky sessions through getStickyWeightedExecutionKey, which generates a composite key from the combo name and configuration. If the current target remains healthy, the engine moves it to the front position and treats subsequent requests as priority routing for the duration of the stickyWeightedLimit. Success counters update via recordStickyWeightedSuccess to track the current streak.

Can strategies be combined or switched dynamically?

While individual combos specify a single strategy via the strategy field, the auto strategy enables dynamic selection by evaluating live metrics (latency, cost, success rates) through resolveAutoStrategyOrder at line 1087. This effectively combines multiple strategies by selecting the most appropriate algorithm per-request based on current system conditions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}