OmniRoute Routing Strategies: Complete Guide to 19 Dispatch Methods and When to Use Each

OmniRoute exposes 19 distinct routing strategies through the ROUTING_STRATEGY_VALUES constant in src/shared/constants/routingStrategies.ts, enabling dispatch algorithms from simple priority queues to intelligent context-aware selection and multi-provider fusion.

OmniRoute's combo engine routes requests across its provider catalog using configurable dispatch logic defined in the routing strategies constants file. These strategies determine how the system selects target models for each request, balancing constraints like cost, latency, context window size, and load distribution. Understanding the specific behavior of each strategy is essential for optimizing performance and cost across your AI pipeline.

Basic Distribution Strategies

These foundational strategies provide predictable traffic patterns for straightforward load sharing or priority-based failover.

Priority

Always picks the first (highest-priority) target in the configuration list.

Use priority routing when you have a definitive primary provider that must handle all traffic unless it becomes unavailable, with secondary providers acting as strict fallbacks. This creates a deterministic failover chain where requests only cascade to lower-priority targets upon upstream failure.

Weighted

Selects targets based on assigned numerical weights.

Use weighted routing when you need proportional traffic distribution—for example, sending 70% of requests to a high-performance provider and 30% to a budget alternative. The combo engine normalizes these weights to ensure statistical distribution matches your defined ratios over time.

Round-Robin

Cycles through targets in a fixed sequential order.

Use round-robin for uniform load distribution across equally-capable providers. This prevents statistical bias and ensures no single target receives disproportionate request volume, making it ideal for stateless workloads where provider capability is homogeneous.

Random and Strict-Random

Random selects targets uniformly at random, suitable for stateless requests where temporal distribution isn't critical. Strict-random enforces stricter fairness guarantees to prevent statistical clustering that can occur with pure random selection over short time windows.

Use strict-random when you need guaranteed temporal evenness across small sample sizes.

Load-Aware and Capacity Strategies

These algorithms monitor provider capacity, usage quotas, and current load to make intelligent routing decisions.

Power-of-Two-Choices (p2c)

Samples two random targets and selects the one with lower current load.

Use p2c to avoid the "thundering herd" problem while maintaining low overhead. This algorithm provides near-optimal load balancing without requiring global state knowledge, making it efficient for high-throughput scenarios.

Least-Used

Prefers the target with the lowest recent usage counters.

Use least-used when you want to spread load to the provider that has been utilized least recently. This strategy works well for sticky-session-like behavior where recent inactivity implies availability.

Headroom

Chooses targets with the most remaining quota headroom.

Use headroom when managing API rate limits across multiple providers to prevent exhausting any single provider's quota prematurely. This maximizes your effective throughput across the provider pool.

Reset-Aware and Reset-Window

Reset-aware prefers targets that have recently reset their usage counters (e.g., at midnight or billing cycle boundaries). Reset-window implements a sliding-window policy for quota management.

Use these strategies for time-boxed quota optimization—specifically when you want to maximize utilization of providers that have freshly replenished their limits or when implementing rolling quota windows to avoid hard cutoffs.

Fill-First

Fills the first target completely until reaching a defined limit, then moves to the next target in sequence.

Use fill-first for tiered consumption models, such as exhausting a free tier completely before moving to paid tiers, or consuming reserved capacity before on-demand resources.

Cost and Context Optimization

These strategies optimize for economic efficiency and technical constraints like token limits and caching behavior.

Cost-Optimized

Selects the cheapest target that satisfies the request requirements.

Use cost-optimized for batch processing, offline jobs, or non-latency-sensitive workloads where minimizing API spend takes precedence over response time. The engine evaluates current pricing across compatible providers before dispatch.

Context-Optimized

Prioritizes providers that can efficiently handle the current context size.

Use context-optimized when processing large input windows to avoid providers with limited context capacity or those that charge premium rates for large context processing. This prevents context-window overflow errors and minimizes per-token costs for long conversations.

Cache-Optimized

Prefers providers that benefit from cached results or exhibit better cache hit rates for specific query patterns.

Use cache-optimized for repetitive queries where certain providers offer lower latency or cost for cached content, improving response times and reducing redundant computation.

Context-Relay

Relays the request to the next target while preserving full conversation context.

Use context-relay for stateful conversations that need to transition between providers mid-stream without losing conversational history. This enables provider switching within ongoing chat sessions while maintaining continuity.

Intelligent and Adaptive Strategies

These advanced strategies require runtime metric collection and historical performance data.

Auto

The built-in Auto Combo strategy that dynamically selects the best target based on real-time metrics including latency, error rates, and cost.

Use auto when you want OmniRoute to optimize routing without manual configuration of weights or priorities. The engine continuously evaluates provider performance and adjusts dispatch decisions accordingly.

Last-Known-Good-Provider (lkgp)

Falls back to the last provider that successfully handled the same request type.

Use lkgp for resilient failover scenarios where recent success is the strongest predictor of future availability. This creates self-healing routes that gravitate toward currently reliable providers.

Multi-Provider Composition Strategies

These strategies involve multiple providers in a single request lifecycle.

Fusion

Combines results from multiple providers into a single unified response.

Use fusion for ensemble methods where you want to aggregate outputs from several models, such as voting mechanisms, consensus building, or blended generations that merge the strengths of multiple providers.

Pipeline

Pipes the output of one provider as input to the next, forming a processing pipeline.

Use pipeline for multi-stage workflows like initial classification followed by specialized generation, preprocessing before main inference, or chaining models with complementary capabilities.

Internal Strategies

Quota-Share

An internal-only strategy used by automatically generated combos for quota sharing. Quota-share is not exposed in the UI or public API, but is enumerated in src/shared/constants/routingStrategies.ts for system-level quota management across provider pools.

Working with Routing Strategies in Code

The routing strategies are accessible throughout the OmniRoute codebase via the constants file and associated helper utilities.

Listing Available Strategies

To retrieve the complete list of valid strategies for UI display or validation:

import { ROUTING_STRATEGY_VALUES } from '@/shared/constants/routingStrategies';

// Print the full list of 19 strategies
console.log('Available routing strategies:', ROUTING_STRATEGY_VALUES);

Creating a Combo with Specific Strategy

When persisting combo configurations to the database via src/lib/db/combo.ts:

import { createCombo } from '@/lib/db/combo';
import { ROUTING_STRATEGY_VALUES } from '@/shared/shared/constants/routingStrategies';

async function makeCombo() {
  const combo = await createCombo({
    name: 'cost-efficient-combo',
    strategy: 'cost-optimized',  // Any value from ROUTING_STRATEGY_VALUES
    targets: [{ provider: 'openai', model: 'gpt-4' }],
  });
  console.log('Combo created with strategy:', combo.strategy);
}

Normalizing User Input

The helper function normalizeRoutingStrategy resolves user-friendly inputs to canonical strategy names:

import { normalizeRoutingStrategy } from '@/shared/constants/routingStrategies';

const userInput = 'Cost';
const strategy = normalizeRoutingStrategy(userInput);
console.log('Normalized strategy:', strategy); // → "cost-optimized"

Key Source Files

Understanding where these strategies are implemented helps when extending or debugging routing behavior:

• src/shared/constants/routingStrategies.ts – Canonical enumeration of all 19 user-facing strategies plus internal quota-share, plus normalization utilities
• src/app/(dashboard)/dashboard/combos/page.tsx – Dashboard UI component for displaying and editing combo strategies
• open-sse/services/combo/comboSetup.ts – Service layer importing strategy constants to configure the combo engine at runtime
• src/lib/db/combo.ts – Database persistence layer for combo configurations, referencing the strategy values

Summary

• OmniRoute defines 19 user-facing routing strategies in src/shared/constants/routingStrategies.ts, plus one internal quota-share strategy
• Basic strategies (priority, weighted, round-robin) provide predictable failover and distribution
• Load-aware strategies (p2c, headroom, fill-first) optimize for capacity constraints and quota management
• Cost strategies minimize spend while context strategies prevent token limit errors
• Advanced strategies like fusion and pipeline enable multi-provider composition for complex workflows
• The ROUTING_STRATEGY_VALUES constant exposes valid strategies to the UI, API, and combo configuration

Frequently Asked Questions

How do I choose the right routing strategy for my use case?

Select priority for strict failover chains, weighted for proportional traffic splitting, and auto when you want OmniRoute to optimize based on real-time metrics. Use cost-optimized for batch jobs, context-optimized for large token windows, and fusion when you need ensemble results from multiple models.

Can I combine multiple routing strategies in a single combo?

Individual combos reference one primary strategy, but you can achieve complex behavior by using fusion or pipeline strategies, which inherently involve multiple providers. For sequential logic, nest combos where the output of one routed request feeds into another combo with different strategy settings.

What is the difference between the auto and cost-optimized strategies?

Cost-optimized statically selects the cheapest available provider at request time, while auto dynamically weighs multiple runtime factors including latency, error rates, and cost. Use cost-optimized when price is the absolute priority; use auto when you need balanced optimization across cost, speed, and reliability.

Why is quota-share not available in the dashboard UI?

Quota-share is reserved for internal system use by automatically generated combos that manage quota allocation across provider pools. It is intentionally excluded from ROUTING_STRATEGY_VALUES exposure in the public API to prevent manual configuration conflicts with OmniRoute's automated quota management system.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}