When to Use Each OmniRoute Routing Strategy: A Complete Guide to 19 Built-In Policies

Use the priority strategy for deterministic fallback ordering, cost-optimized for budget-sensitive workloads, lkgp for session continuity, and the default auto strategy for general-purpose routing that automatically balances cost, latency, and reliability.

OmniRoute provides 19 built-in routing strategies declared in src/shared/constants/routingStrategies.ts, each implementing a different load-balancing or selection policy. This guide maps specific use cases to the appropriate strategy, with direct references to source code implementations and practical configuration examples.

---

Deterministic and Tier-Based Strategies

These strategies provide predictable, rule-based routing for subscription management and strict ordering requirements.

priority: Strict Hierarchical Fallback

Use priority when you need a deterministic order—such as exhausting a primary provider before trying secondary options. Ideal for subscription-first models where you want to consume paid quota before pay-as-you-go alternatives.

Implementation reference: ROUTING_STRATEGY_VALUES at src/shared/constants/routingStrategies.ts#L2-L4

{
  "name": "enterprise-tier",
  "strategy": "priority",
  "targets": [
    { "model": "azure/gpt-4o" },
    { "model": "openai/gpt-4o" }
  ]
}

fill-first: Quota Exhaustion Before Advancement

Use fill-first to completely drain one provider's quota before moving to the next. Perfect for "use up my Azure commitment before paying OpenAI retail" scenarios.

context-relay: Handoff-Aware Priority

Use context-relay for long-running conversations where you need to hand off context after a quota warning. Behaves like priority but injects a handoff summary to maintain conversation continuity.

---

Load Distribution and Randomization Strategies

These strategies spread traffic across providers using probability or rotation rather than strict ordering.

weighted: Probabilistic Mix for A/B Testing

Use weighted for gradual rollouts or A/B testing new providers. Assign per-target weights to control traffic proportion.

round-robin: Even Distribution Without Weighting

Use round-robin for simple, fair load-balancing across providers or debugging multiple backends with predictable rotation.

p2c: Power-of-Two-Choices Load Balancing

Use p2c (Power-of-Two-Choices) for high-throughput services needing to avoid hot spots. Randomly selects two candidates and prefers the less-loaded option—a proven technique from distributed systems research.

random and strict-random: True Uniform Selection

Use random for stress-testing or when you genuinely don't care about any metric. Use strict-random when you need independent random seeds per request—even across very short intervals—with no deduplication of repeats.

---

Cost and Performance Optimization

These strategies optimize for specific operational metrics: cost, latency, or resource headroom.

cost-optimized: Cheapest Provider First

Use cost-optimized for batch jobs, background processing, or any cost-sensitive workload. Selects based on live pricing data.

{
  "name": "batch-processor",
  "strategy": "cost-optimized",
  "targets": [
    { "model": "openai/gpt-4o-mini" },
    { "model": "google/gemini-1.5-flash" }
  ]
}

least-used: Lowest Current Request Count

Use least-used when you have many similar providers and want to spread active load evenly.

headroom: Maximum Remaining Quota

Use headroom when you need to guarantee large requests succeed without hitting limits. Selects the provider with the most remaining quota headroom.

---

Quota Temporal Awareness

These strategies make intelligent decisions based on quota reset timing.

reset-aware: Prefer Near-Reset Providers ⭐

Use reset-aware when you want to finish requests before a quota reset to avoid throttling. The scoring algorithm prefers providers whose reset window is approaching.

reset-window: Strict Nearest-Reset Ordering

Use reset-window for workloads that can tolerate slight latency spikes in exchange for better quota utilization. Strictly orders by nearest reset time rather than scoring.

---

Auto-Combo: The Recommended Default

auto: 9-Factor Intelligent Scoring

Use auto as your general-purpose strategy. It runs the Auto-Combo scoring engine (open-sse/services/autoCombo/scoring.ts) with 9 weighted factors and automatically picks the best candidate per request.

Zero-configuration usage:

{
  "model": "auto",
  "messages": [{ "role": "user", "content": "Explain quantum computing" }]
}

Available variants:

• auto/fast — prioritizes latency
• auto/cheap — prioritizes cost
• auto/reliable — prioritizes uptime

Per-request steering via headers:

curl -X POST http://localhost:20128/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "X-OmniRoute-Mode: fast" \
  -H "X-OmniRoute-Budget: 0.02" \
  -d '{"model":"auto","messages":[...]}'

Auto-Combo router strategies (from open-sse/services/autoCombo/routerStrategy.ts):

• rules — custom rule-based matching
• cost — cost-only optimization
• latency — fastest response time
• sla-aware — balances latency, error rate, and cost for strict SLOs
• lkgp — Last-Known-Good-Provider for cache stickiness

The auto strategy builds candidate pools on-the-fly via open-sse/services/autoCombo/virtualFactory.ts—no database entry required.

---

Session and Context Optimization

lkgp: Last-Known-Good-Provider

Use lkgp for multi-turn conversation continuity. Keeps the same provider across turns for cache stickiness, then falls back to priority ordering. Implemented in open-sse/services/autoCombo/routerStrategy.ts.

context-optimized: Window Size Matching

Use context-optimized for large-context tasks like long document processing. Chooses the provider whose context window best matches the request's required size.

cache-optimized: Prompt Cache Affinity

Use cache-optimized when you have heavy prompt caching enabled. Prioritizes connections likely to already hold the prompt-cache prefix, reducing recompute. Implemented in open-sse/services/combo/promptCacheAffinity.ts.

---

Advanced Processing Strategies

fusion: Multi-Model Parallel Fan-Out 🧬

Use fusion for highest-quality answers requiring multiple perspectives. Fans out to a panel of models in parallel and synthesizes a single answer via a judge model.

{
  "name": "quality-panel",
  "strategy": "fusion",
  "targets": [
    { "model": "openai/gpt-4o-mini" },
    { "model": "anthropic/claude-3-opus-100k" },
    { "model": "google/gemini-1.5-flash" }
  ],
  "config": {
    "judgeModel": "openai/gpt-4o-mini",
    "fusionTuning": { "minPanel": 2, "stragglerGraceMs": 5000 }
  }
}

Implementation: open-sse/services/fusion.ts

pipeline: Sequential Multi-Stage Workflows

Use pipeline for multi-stage processing like generate → refine → summarize. Runs targets sequentially, feeding each step's output into the next.

---

Quick Decision Matrix

Goal	Strategy
Deterministic fallback order	priority or fill-first
Probabilistic mix / A/B testing	weighted or p2c
Even rotation	round-robin
Lowest cost	cost-optimized
Fastest latency	latency (via auto)
Strict SLO compliance	sla-aware (auto router)
Session stickiness	lkgp
Large context windows	context-optimized
Prompt cache efficiency	cache-optimized
Multi-model quality	fusion
Sequential workflows	pipeline
Quota reset awareness	reset-aware or reset-window
General purpose	auto (recommended default)

---

Summary

• 19 strategies are declared in src/shared/constants/routingStrategies.ts, covering deterministic ordering, probabilistic distribution, cost optimization, temporal quota awareness, and advanced multi-model processing.
• auto is the recommended default for most use cases, providing intelligent 9-factor scoring without configuration.
• priority and fill-first solve subscription management and tier-based routing needs.
• cost-optimized and cost (auto router) target budget-sensitive workloads.
• lkgp and cache-optimized optimize for session continuity and prompt cache efficiency.
• fusion and pipeline enable advanced answer synthesis and multi-stage workflows.
• Per-request headers (X-OmniRoute-Mode, X-OmniRoute-Budget) allow runtime steering of even zero-config auto combos.

---

Frequently Asked Questions

What is the difference between reset-aware and reset-window?

reset-aware uses a scoring approach that prefers providers with approaching reset times, balancing this factor against others. reset-window strictly orders providers by nearest reset time, making it more aggressive about quota utilization at potential latency cost. Use reset-aware for balanced optimization and reset-window when quota efficiency is paramount.

Can I use multiple strategies together?

You can achieve composite behavior through persisted combos (POST /api/combos) that embed one strategy, or by using auto variants and per-request headers for runtime steering. The fusion and pipeline strategies inherently combine multiple targets. For custom logic, the auto router supports rule-based overrides in open-sse/services/autoCombo/routerStrategy.ts.

How do I debug which provider was selected?

The auto strategy and most persisted combos expose routing metadata in response headers. Additionally, OmniRoute's SSE handler in src/sse/handlers/chat.ts logs selection decisions when debug logging is enabled. For fusion strategies, the response includes individual panel member outputs and the judge model's synthesis reasoning.

When should I use auto versus a persisted combo with a specific strategy?

Use auto for dynamic, metric-driven selection without operational overhead—ideal for most production traffic. Use a persisted combo when you need deterministic behavior (e.g., priority for compliance), specialized processing (e.g., fusion for quality), or explicit target lists with custom configuration that should be version-controlled and reusable across requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}