How OmniRoute's Combo Routing Engine Handles 17 Different Routing Strategies Including Priority, Weighted, and Round-Robin

OmniRoute processes requests through a unified pipeline where handleComboChat extracts the strategy type, resolves targets into a flat array of ResolvedComboTarget objects, and delegates ordering to the pure function applyStrategyOrdering, which implements distinct algorithms ranging from simple priority queues to distributed deficit round-robin with power-of-two-choices selection.

The diegosouzapw/OmniRoute repository implements a sophisticated multi-strategy routing system that allows developers to define "combos"—groups of AI models with specific routing behaviors. At the heart of this system lies the combo routing engine, which processes 17 different strategies through a modular architecture that separates target resolution from execution ordering.

Core Routing Architecture

The engine follows a four-stage pipeline defined in open-sse/services/combo.ts:

1. Strategy Dispatch (lines 773-807): handleComboChat extracts the strategy field from the combo definition. Special strategies like "fusion" and "pipeline" branch to dedicated handlers, while standard strategies proceed through the generic combo pipeline.

2. Target Resolution (around line 6575): resolveComboTargets expands provider wildcards, connection strings, and fingerprints into a flat array of ResolvedComboTarget objects.

3. Strategy Ordering: For non-auto strategies, applyStrategyOrdering in open-sse/services/combo/applyStrategyOrdering.ts reorders the target list according to the selected algorithm without side effects.

4. Execution: executeRuntimeUnitCombo (lines 858-901) traverses the ordered list sequentially, applying per-target timeouts, retry logic, and fallback handling.

Sequential Execution Strategies

Priority Strategy

The default "first-in-list" approach maintains the exact order targets appear in the combo configuration. The applyStrategyOrdering function performs no reordering for "priority" strategies. However, handleComboChat implements a unique pre-screen at lines 12004-12011 that checks latency and health for all priority targets, allowing the engine to skip unhealthy early targets without waiting for full request timeouts.

Fill-First Strategy

Maintains the original priority order but stops execution once a target succeeds—useful when a provider must "fill" a specific quota before falling back to alternatives. The implementation logs the strategy selection but performs no reordering (lines 120-124 in applyStrategyOrdering.ts).

Load Distribution Strategies

Weighted Strategy with Sticky Sessions

Distributes traffic proportionally to assigned weights while supporting sticky selection through getStickyWeightedExecutionKey. The implementation:

• Derives a sticky key based on the combo name and configured stickyWeightedLimit (lines 858-874 in combo.ts)
• Checks if the sticky target remains eligible; if so, moves it to the front and falls back to priority ordering for remaining targets
• Shuffles remaining targets with fisherYatesShuffle before weighted selection
• Records successful sticky executions via recordStickyWeightedSuccess (lines 938-940)

Round-Robin Strategy

Rotates through targets sequentially while supporting sticky sessions via rrStickyTargets. The getStickyRoundRobinStartIndex calculates the starting position based on the configured stickyRoundRobinLimit. If the limit exceeds 1, the algorithm rotates the array accordingly; otherwise, it simply increments a counter (lines 888-904 in combo.ts). Success updates the state via recordStickyRoundRobinSuccess (lines 945-954).

Random and Strict-Random Strategies

Random performs a fresh Fisher-Yates shuffle on every request (lines 117-119 in applyStrategyOrdering.ts). Strict-Random combines deterministic deck-based selection with random fallbacks: it uses a persistent deck key (combo:<name>) via getNextFromDeck (from src/shared/utils/shuffleDeck.ts), places the selected target first, then shuffles the remaining targets (lines 96-112).

Cost and Capacity Optimization Strategies

Cost-Optimized Strategy

Ranks targets by estimated cost per token using sortTargetsByCost. When manifestRouting is enabled, the algorithm applies additional pricing hints from the catalog (lines 131-172 in applyStrategyOrdering.ts).

Context-Optimized Strategy

Selects the model with the largest context window first by calling sortTargetsByContextSize, ensuring requests requiring many tokens hit capable providers initially (lines 196-199).

Headroom Strategy

Prioritizes targets with the most available capacity according to internal usage metrics. The orderTargetsByHeadroom function performs this ranking (lines 199-210).

Least-Used Strategy

Balances load by favoring targets with the lowest invocation count. The sortTargetsByUsage function from targetSorters.ts handles the reordering (lines 128-131).

Quota and Reset-Aware Strategies

Reset-Aware and Reset-Window Strategies

Prefer targets whose quota reset windows are closest to replenishment. orderTargetsByResetAwareQuota and orderTargetsByResetWindow implement these algorithms (lines 172-196).

Quota-Share Strategy

Implements a distributed deficit round-robin (DRR) combined with power-of-two-choices (P2C) selection. The selectQuotaShareTarget in quotaShareStrategy.ts respects per-connection concurrency limits resolved via resolveMaxConcurrentByConnection (lines 210-223 in applyStrategyOrdering.ts).

Dynamic Strategy Selection

Auto Strategy

Dynamically selects routing algorithms based on live metrics including latency, success rate, and cost. Delegated to resolveAutoStrategyOrder in resolveAutoStrategy.ts, this module builds candidate objects, scores them, and returns an ordered list (starting at line 1087 in combo.ts).

Configuration Examples

Defining a Weighted Strategy Combo

{
  "name": "my-weighted-combo",
  "strategy": "weighted",
  "config": {
    "stickyWeightedLimit": 3
  },
  "models": [
    "openai/gpt-4o-mini",
    "anthropic/claude-3.5-sonnet",
    "groq/llama-3.1-70b"
  ]
}

When invoked, OmniRoute uses the sticky-weighted key (max 3 consecutive selections) if the target remains healthy; otherwise, it shuffles targets and selects proportionally to implicit weights.

Round-Robin with Sticky Sessions

{
  "name": "rr-combo",
  "strategy": "round-robin",
  "config": {
    "stickyRoundRobinLimit": 5
  },
  "models": [
    "openai/gpt-4o",
    "openai/gpt-4o-mini",
    "anthropic/claude-3-opus"
  ]
}

Each request rotates through the three targets, staying on the same target for up to five successive requests before moving to the next index.

Priority Configuration

{
  "name": "priority-combo",
  "strategy": "priority",
  "models": [
    "openai/gpt-4o-mini",
    "anthropic/claude-3-sonnet"
  ]
}

Targets are tried exactly in list order; the pre-screen step may skip unhealthy targets without waiting for timeouts.

Summary

• Centralized Ordering: Strategy logic lives in applyStrategyOrdering.ts, while execution state management resides in handleComboChat within combo.ts
• 17 Distinct Algorithms: Range from simple priority queues to complex quota-aware distributed round-robin with concurrency limits
• Sticky Session Support: Both weighted and round-robin strategies implement sticky selection through dedicated key generation functions and state tracking
• Special Handling: Fusion and pipeline strategies bypass the standard ordering pipeline entirely for custom request handling
• Pure Functions: The ordering phase is side-effect-free, making strategies testable and deterministic given the same target list

Frequently Asked Questions

What is the difference between Random and Strict-Random strategies in OmniRoute?

Random performs a fresh Fisher-Yates shuffle on every request using fisherYatesShuffle (lines 117-119), while Strict-Random uses a persistent deck system via getNextFromDeck (lines 96-112) to ensure deterministic cycling through targets before reshuffling, providing more uniform distribution across multiple requests.

How does OmniRoute handle sticky sessions in weighted routing?

The engine generates a sticky key using getStickyWeightedExecutionKey based on the combo name and configured limit (lines 858-874). If the keyed target remains healthy, it moves to the front of the queue; otherwise, the system falls back to shuffled weighted selection. Success increments a counter via recordStickyWeightedSuccess (lines 938-940).

Where is the routing strategy actually applied in the OmniRoute codebase?

Strategy ordering occurs in open-sse/services/combo/applyStrategyOrdering.ts, a pure function called by handleComboChat in open-sse/services/combo.ts. Execution happens in executeRuntimeUnitCombo, which traverses the ordered list with fallback logic and per-target timeouts.

Can OmniRoute automatically select the best routing strategy for my combo?

Yes, the "auto" strategy delegates to resolveAutoStrategyOrder in resolveAutoStrategy.ts, which scores candidates based on real-time metrics including latency, cost, and success rates, then returns an optimized target ordering without manual configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}