OmniRoute’s 19 Routing Strategies: Complete Guide with Code Examples

OmniRoute’s 19 routing strategies are deterministic or probabilistic rules defined in ROUTING_STRATEGY_VALUES that control how requests are distributed across AI providers and models, ranging from simple load-balancing methods to advanced multi-model orchestration patterns.

OmniRoute is an open-source AI gateway that routes every request through a combo engine. The engine selects one or more provider/model targets based on a configurable routing strategy. These strategies live in src/shared/constants/routingStrategies.ts and are validated through Zod schemas before execution. Understanding all 19 strategies lets you optimize for cost, latency, reliability, or answer quality depending on your workload.

---

Where the 19 Routing Strategies Are Defined

All publicly available strategies are declared as the constant ROUTING_STRATEGY_VALUES in src/shared/constants/routingStrategies.ts. This array contains exactly 19 entries that the combo engine recognizes.

The same list appears in the official documentation at docs/routing/AUTO-COMBO.md. An additional internal-only strategy, quota-share, exists as INTERNAL_ROUTING_STRATEGY_VALUES but is never exposed to users or the UI.

Zod validation happens in src/shared/validation/schemas/combo.ts, which builds a strict enum from ROUTING_STRATEGY_VALUES to prevent invalid strategy values at runtime.

---

Load-Balancing and Distribution Strategies

These six strategies handle basic traffic distribution across eligible targets.

priority

Uses a static ordered list of targets. The engine tries the first target; if unavailable, it falls back to the second, and so on.

Implementation: open-sse/services/combo/targetResolution.ts (ordered-list fallback logic)

Best for: Guaranteed provider preference with manual failover.

weighted

Randomly selects a target based on per-target weight values. Higher weights receive proportionally more traffic.

Implementation: open-sse/services/combo/targetSorters.ts (weight-based sorter)

Best for: Gradual traffic shifting or capacity-proportional distribution.

round-robin

Cycles through targets sequentially using a persistent pointer that advances with each request.

Implementation: open-sse/services/combo/targetResolution.ts (RR state tracking)

Best for: Even distribution without randomization.

random

Uniform random selection among all eligible targets.

Implementation: open-sse/services/combo/random.ts

Best for: Simple, stateless distribution.

strict-random

Uniform random selection without deduplication of recent choices. The same target can be selected multiple times in succession.

Implementation: open-sse/services/combo/strictRandom.ts

Best for: True randomness where temporal clustering is acceptable.

p2c (Power-of-2-Choices)

Picks two random targets and selects the one with lower current load. This reduces tail latency compared to pure random selection.

Implementation: open-sse/services/combo/p2c.ts

Best for: Load-sensitive balancing with minimal coordination overhead.

---

Quota and Cost Optimization Strategies

These five strategies make routing decisions based on provider limits, pricing, and usage patterns.

fill-first

Exhausts a target's quota completely before moving to the next target in sequence.

Implementation: open-sse/services/combo/fillFirst.ts

Best for: Maximizing utilization of prepaid or limited-availability tiers.

least-used

Selects the target with the fewest active requests at decision time.

Implementation: open-sse/services/combo/leastUsed.ts

Best for: Preventing hotspot formation on popular providers.

cost-optimized

Minimizes dollar cost per request using catalog pricing data for each provider/model combination.

Implementation: open-sse/services/combo/costOptimized.ts

Best for: Budget-conscious workloads with flexible quality requirements.

reset-aware

Prioritizes targets whose quota reset window is shortest — those getting fresh quota soonest.

Implementation: open-sse/services/combo/resetAware.ts

Best for: Near-depleted quotas where waiting is cheaper than switching.

reset-window

Prefers targets with the nearest reset timestamp, regardless of current usage level.

Implementation: open-sse/services/combo/resetWindow.ts

Best for: Predictable quota cycling across time zones or billing periods.

headroom

Picks the target with the largest remaining quota headroom (most capacity remaining).

Implementation: open-sse/services/combo/headroom.ts

Best for: Avoiding providers approaching rate limits.

---

Context and State-Aware Strategies

These three strategies incorporate conversation history, caching, and previous success patterns.

context-relay

Passes conversation context from one target to the next, enabling long-running dialogues across different models. The full message history travels with each hop.

Implementation: open-sse/services/combo/contextRelay.ts

Best for: Multi-turn conversations where provider switching mid-dialog is required.

context-optimized

Selects the target best suited to the current request's context size, matching large contexts to providers with generous context windows.

Implementation: open-sse/services/combo/contextOptimized.ts

Best for: Variable-length prompts where context limits vary by provider.

cache-optimized

Reorders targets based on prompt-cache affinity. The provider most likely to have the request's prefix already cached is tried first.

Implementation: open-sse/services/combo/promptCacheAffinity.ts

Best for: Repeated similar prompts where cache hits reduce latency and cost.

---

Reliability and Recovery Strategies

These two strategies improve success rates through pinning and fallback behavior.

lkgp (Last-Known-Good-Path)

Pins to the last successful provider for subsequent requests from the same session. Falls back to rule-based routing only on failure.

Implementation: open-sse/services/combo/lkgp.ts

Best for: Session stability where provider switching causes disruption.

---

Advanced Multi-Model Strategies

These two strategies invoke multiple models and combine or chain their outputs.

fusion

Fans out the request to a panel of models in parallel, then synthesizes a final answer using a dedicated judge model that evaluates and merges responses.

Implementation: open-sse/services/fusion.ts

Best for: Maximum answer quality through ensemble reasoning.

pipeline

Executes targets sequentially, passing each step's output as the next step's input. Only the final step's answer is returned to the caller.

Implementation: open-sse/services/pipeline.ts

Best for: Multi-stage processing like drafting → editing → fact-checking.

---

Recommendation Engine Strategy

auto

The Auto-Combo engine — OmniRoute's recommended default. Computes a 15-factor score balancing cost, latency, stability, provider health, context fit, and more to select the optimal target dynamically.

Implementation: open-sse/services/autoCombo/* (distributed across scoring modules)

Best for: General-purpose routing without manual tuning.

---

How to Configure Routing Strategies

Method 1: Persisted Combo Definition

Store the strategy permanently when creating a combo via API:

// POST /api/combos
await fetch('http://localhost:20128/api/combos', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    name: 'Production GPT-4 Cluster',
    strategy: 'weighted',
    models: [
      { provider: 'openai', model: 'gpt-4o', weight: 70 },
      { provider: 'azure', model: 'gpt-4o', weight: 30 }
    ]
  })
});

The strategy field is strictly validated against ROUTING_STRATEGY_VALUES in src/shared/validation/schemas/combo.ts.

Method 2: Per-Request Header Override

Override the persisted strategy for a single request using the X-OmniRoute-Strategy header:

curl -X POST http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-OmniRoute-Strategy: fusion" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Analyze this contract"}]
  }'

This header is processed in open-sse/services/autoCombo/requestControls.ts and forces the specified strategy for that request only.

Method 3: Programmatic Invocation

Manually invoke the resolution engine with a specific strategy:

import { resolveComboTarget } from '@/open-sse/services/combo/targetResolution';

const combo = await getComboById('production-cluster');
const target = await resolveComboTarget(combo, { 
  strategy: 'least-used'  // Override for this call
});

console.log('Selected:', target.provider, target.model);

resolveComboTarget delegates to strategyDispatch.ts, which maps the strategy string to the appropriate sorter implementation.

---

Strategy Decision Flow

The combo engine determines which strategy to apply through this precedence:

1. Request header override (X-OmniRoute-Strategy) — highest priority
2. Persisted combo strategy — stored in combo definition
3. Auto-Combo computation — when strategy is "auto"

This hierarchy allows flexible override patterns without requiring combo reconfiguration.

---

Summary

• OmniRoute exposes 19 routing strategies declared in src/shared/constants/routingStrategies.ts with Zod validation in src/shared/validation/schemas/combo.ts.

• Distribution strategies (priority, weighted, round-robin, random, strict-random, p2c) handle basic load balancing with varying randomization and statefulness.

• Quota strategies (fill-first, least-used, cost-optimized, reset-aware, reset-window, headroom) optimize for budget, capacity, and rate-limit management.

• Context strategies (context-relay, context-optimized, cache-optimized) preserve conversation state and exploit provider-specific caching.

• Reliability strategies (lkgp, auto) improve success rates through pinning and multi-factor scoring.

• Orchestration strategies (fusion, pipeline) enable multi-model parallel and sequential execution.

• Strategy selection follows header → persisted → auto precedence, with all values dispatched through open-sse/services/combo/strategyDispatch.ts.

---

Frequently Asked Questions

What is the default routing strategy in OmniRoute?

The default strategy is auto, which engages the Auto-Combo engine. This strategy computes a 15-factor score across all available providers and selects the optimal target based on real-time cost, latency, stability, and capacity signals. It requires no manual configuration and adapts automatically to changing conditions.

Can I combine multiple routing strategies in one combo?

No — each combo specifies exactly one strategy at persistence time. However, you can achieve composite behavior through strategy-specific mechanisms: fusion and pipeline inherently use multiple models, while auto internally combines multiple factors. For dynamic strategy switching, use the X-OmniRoute-Strategy header to override per-request.

How do I monitor which strategy was used for a request?

OmniRoute includes the resolved strategy in response headers and logs. The open-sse/services/combo/strategyDispatch.ts module records the final strategy selection, which appears in access logs and can be forwarded to external observability systems. Check your deployment's log configuration for the omni_route_strategy field.

What happens if I specify an invalid routing strategy?

The request fails at validation time. src/shared/validation/schemas/combo.ts defines a strict Zod enum derived from ROUTING_STRATEGY_VALUES. Any strategy string not in the 19-entry list triggers a 400 Bad Request with a clear error message listing valid options.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}