OmniRoute I9 Engine: The 19 Combo Routing Strategies Explained

OmniRoute's I9 combo engine supports 19 deterministic and probabilistic routing strategies—including priority, weighted, round-robin, p2c, lkgp, fusion, and auto—that dynamically distribute LLM requests across multiple models based on cost, latency, quota availability, and contextual affinity.

The diegosouzapw/OmniRoute project implements sophisticated traffic routing through its combo (I9) engine, defined primarily in open-sse/services/combo.ts. This system allows developers to specify exactly how requests traverse pools of LLM providers, balancing reliability against performance. The engine enumerates its supported strategies at the top of the core service file, providing granular control over failover, load balancing, and parallel inference.

Core Strategy Categories

The 19 routing strategies fall into four functional categories, each optimized for specific operational requirements.

Deterministic Selection Strategies

These strategies provide predictable, rule-based routing for straightforward failover and sequencing:

• priority: Selects the highest-priority target available in the model list. Supports a "fallback-only-on-quota-exhaustion" flag to prevent premature failovers when quotas are temporarily exhausted.
• round-robin: Cycles through targets in sequential order, optionally respecting native Codex turn-pinning for session sticky behavior.
• fill-first: Attempts to saturate the first target that possesses sufficient quota before considering subsequent models, ideal for draining credits on specific providers.
• random: Chooses a target uniformly at random from the available pool.
• strict-random: A variant that deliberately eliminates any selection bias from previous attempts, ensuring cryptographic-grade randomness for compliance scenarios.

Load Balancing and Performance Strategies

These algorithms distribute traffic based on real-time system state and historical performance metrics:

• weighted: Routes requests proportionally to assigned weight values. Supports sticky-weighted limits to prevent thrashing between high-weight targets.
• least-used: Selects the target with the smallest recent usage count, preventing hot-spotting on popular models.
• p2c (power-of-two-choices): Samples two random targets and applies a scoring function to select the superior option, reducing load variance compared to pure random selection.
• lkgp (least-known-good-performance): Routes to the target exhibiting the lowest historical latency and error rate according to comboMetrics.ts data.
• cost-optimized: Prioritizes targets based on current pricing data, automatically routing to the cheapest available provider.

Context and State-Aware Strategies

These methods incorporate session history and provider rate-limit state into routing decisions:

• context-optimized: Scores targets by how closely they match the current session's context affinity, ensuring model consistency for multi-turn conversations.
• context-relay: Specifically relays context from a previous turn to a designated target, enabling continuation-style flows across disparate model architectures.
• reset-aware: Considers provider-level reset-window state when calculating target scores, avoiding providers in cooldown.
• reset-window: Applies a reset-window affinity factor to each candidate, respecting hard rate-limit windows defined in the provider configuration.

Dynamic and Parallel Strategies

Advanced patterns for complex workflows and ensemble inference:

• auto: A meta-strategy that dynamically builds candidate pools via buildAutoCandidates, applying quota-share heuristics, context affinity, and concurrency limits without requiring a static model list.
• fusion: Fans out requests to a panel of models in parallel, then employs a judge model to synthesize a single, unified final answer from multiple reasoning paths.

Implementation Architecture

When a request hits the /v1/chat/completions endpoint with a combo configuration, the handleComboChat routine extracts the strategy property and branches to specialized dispatch logic.

Strategy-specific execution paths appear as conditional branches within the main handler:

if (strategy === "round-robin") {
  // Iterate via counters managed in rrState.ts
}
if (strategy === "fusion") {
  // Execute parallel fan-out via fusion.ts
}

For the auto strategy, the system invokes buildAutoCandidates from open-sse/services/combo/autoStrategy.ts. This function respects quotaShareConcurrencyLimit settings and dynamically scores available providers based on real-time availability.

Practical Configuration Examples

Simple priority-based failover:

const combo = {
  name: "my-priority-combo",
  models: [
    { model: "gpt-4o-mini", provider: "openai", weight: 1 },
    { model: "claude-sonnet-4.6", provider: "anthropic", weight: 1 },
  ],
  strategy: "priority",
};

Weighted distribution with sticky limits:

const combo = {
  name: "weighted-rr",
  models: [
    { model: "gemini-2.5-flash", weight: 3 },
    { model: "grok-4-fast", weight: 1 },
  ],
  strategy: "weighted",
};

Auto-scaling with quota protection:

const combo = {
  name: "auto-quota",
  models: [],
  strategy: "auto",
  config: { quotaShareConcurrencyLimit: { enabled: true } },
};

Parallel fusion for ensemble reasoning:

const combo = {
  name: "fusion-example",
  models: [
    { model: "gpt-4o-mini" },
    { model: "claude-opus-4.6" },
  ],
  strategy: "fusion",
};

Supporting Infrastructure

The robustness of these 19 strategies relies on a modular architecture spanning multiple files:

• open-sse/services/combo/autoStrategy.ts: Implements dynamic candidate generation and scoring heuristics for the auto strategy.
• open-sse/services/combo/fusion.ts: Manages parallel execution, response aggregation, and judge-model synthesis for fusion workflows.
• open-sse/services/combo/rrState.ts: Persists round-robin counters and sticky-target state across requests to maintain distribution fairness.
• open-sse/services/combo/comboConfig.ts: Parses strategy-specific configuration options, including weight maps and quota thresholds.
• open-sse/services/combo/comboMetrics.ts: Captures execution telemetry including latency curves and success rates per strategy for the lkgp algorithm.
• open-sse/services/combo/comboCooldownRetry.ts: Implements intelligent retry logic with provider-specific backoff windows, used by reset-aware and reset-window strategies.

Summary

• OmniRoute's I9 engine implements 19 distinct routing strategies in open-sse/services/combo.ts, balancing deterministic routing with probabilistic optimization.
• Deterministic strategies like priority, round-robin, and fill-first provide predictable traffic patterns for simple failover chains.
• Performance strategies including p2c, lkgp, and cost-optimized enable intelligent load distribution based on real-time quota, latency, and pricing data.
• Advanced patterns such as auto and fusion automate candidate selection and parallel inference without manual model list curation.
• The context-aware group (context-optimized, context-relay) preserves conversational continuity and state across heterogeneous model boundaries.
• Supporting infrastructure in rrState.ts, fusion.ts, and comboMetrics.ts provides the state management and observability required for production-grade traffic management.

Frequently Asked Questions

What is the difference between the "auto" and "priority" combo strategies?

The priority strategy follows a strict hierarchical failover chain, routing to the first available model in a predefined list and only proceeding to the next on hard failure. The auto strategy dynamically generates candidates via buildAutoCandidates, weighing factors like quota availability, current latency, and context affinity in real-time without requiring a static model ordering.

How does the "fusion" strategy handle parallel model execution?

The fusion strategy fans out a single request to multiple models simultaneously through open-sse/services/combo/fusion.ts. After receiving all responses, a designated judge model synthesizes the outputs into a single coherent answer, enabling ensemble reasoning at the cost of consuming multiple API quotas per request.

Which strategy is best for minimizing API costs?

Use the cost-optimized strategy to automatically prefer the cheapest available provider based on current pricing data. Alternatively, weighted allows manual cost-based proportioning, while fill-first can exhaust cheaper quotas before falling back to premium models, ensuring budget-conscious routing.

Can I combine strict quota limits with round-robin distribution?

Yes. While round-robin cycles through targets regardless of quota, you can pair quota awareness with distribution by using the weighted strategy with sticky-weighted limits, or rely on the auto strategy which explicitly respects quotaShareConcurrencyLimit settings defined in comboConfig.ts, ensuring hard concurrency ceilings are maintained regardless of distribution algorithm.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}