OmniRoute's 17 Routing Strategies: When to Use Each Load Balancing Method

OmniRoute provides 17 distinct routing strategies—including priority, weighted, round-robin, p2c, cost-optimized, and fusion—that determine how AI requests are distributed across multiple LLM providers based on health, cost, context windows, and usage patterns.

The open-source OmniRoute repository (diegosouzapw/OmniRoute) implements an intelligent request router for LLM APIs. All 17 routing strategies are declared as constants in src/shared/constants/routingStrategies.ts and enforced through Zod schemas in src/shared/validation/schemas/combo.ts, allowing developers to fine-tune exactly how traffic flows between providers like OpenAI, Anthropic, and others.

Where the 17 Strategies Are Defined

The canonical list lives in src/shared/constants/routingStrategies.ts, which exports ROUTING_STRATEGY_VALUES as a TypeScript as const array. This array contains every user-facing strategy, while INTERNAL_ROUTING_STRATEGY_VALUES (containing "quota-share") is reserved for internal workers and never exposed to end-users.

The utility function normalizeRoutingStrategy() in the same file validates incoming strings and defaults to "priority" if an unrecognized value is provided. When the auto-combo engine selects a strategy automatically, it draws from the subset defined in AUTO_ROUTING_STRATEGY_VALUES.

The 17 Routing Strategies and When to Use Them

Failover and Sequential Strategies

priority Uses the first target in the list that reports healthy; only falls back to the next target if the current one fails or returns an error. Best for scenarios where you have a preferred primary provider and only want to use backups during outages.

fill-first Routes all requests to the first target until its quota or rate limit is exhausted, then moves to the second target, and so on. Ideal when you want to maximize usage of specific quota tiers (e.g., using up a limited free tier before falling back to paid alternatives).

Distribution and Load Balancing Strategies

weighted Assigns a numerical weight to each target; requests are distributed proportionally to these weights. Use this when providers have different capacity levels or you want to send 70% of traffic to Provider A and 30% to Provider B.

round-robin Cycles through targets in strict order, giving each provider an equal turn. Suitable for evenly distributing load across similarly-capable providers without skewing toward any particular one.

random Selects a target purely at random from all healthy candidates. Good for uniform stress testing or when you have no preference between equivalent providers.

strict-random Randomly selects a target but never falls back once chosen, even if that target subsequently fails during the request. Use only when you require strict request isolation and cannot tolerate retries across different providers for a single query.

Intelligent Selection Strategies

p2c (Power-of-Two-Choices) Randomly picks two healthy targets, then selects the one with the lower current load or usage counter. This provides better load distribution than pure random selection while avoiding the complexity of full global state tracking.

least-used Routes to the provider with the smallest recent usage counter. Effective when you want to keep utilization metrics balanced across all providers over time.

reset-aware Avoids targets that have recently been reset (e.g., after a server restart or crash) to prevent overwhelming recovering instances. Use in high-availability setups where you want to give providers a "warm-up" period.

reset-window Similar to reset-aware, but respects a configurable time window before considering a reset provider eligible again. Configure this when you know your providers need specific recovery times (e.g., 30 seconds) before accepting full traffic.

lkgp (Last-Known-Good-Provider) Sticky-sessions strategy that routes requests to the provider that most recently succeeded for this user or session. Essential for maintaining consistency in conversational AI where you want to keep a user on the same model instance.

Resource and Cost Optimization Strategies

cost-optimized Selects the cheapest target that satisfies the request's token budget and latency requirements. Best for batch processing or non-latency-sensitive workloads where minimizing API spend is the priority.

headroom Chooses the provider with sufficient remaining token headroom for the specific request size. Critical for routing large context windows (e.g., 128k+ tokens) to providers that won't truncate or reject the payload.

context-optimized Prefers providers offering the largest context windows for big prompts. Use this when you regularly handle large documents or codebases and want to avoid switching to smaller-context models.

Conversation and Context Strategies

context-relay Routes to a provider capable of continuing a conversation using previous context or conversation IDs. Necessary when maintaining multi-turn chat history across provider switches.

Meta and Advanced Strategies

auto Allows the combo dispatcher in open-sse/services/combo/comboSetup.ts to automatically select the most appropriate strategy based on the combo's composition and current provider health. Use when you want hands-off optimization without manually tuning strategy selection.

fusion Sends requests to multiple providers in parallel and merges their responses (experimental). Suitable for critical high-availability scenarios where you want consensus answers or need the fastest possible response time from any available provider.

How to Configure Routing Strategies

Via the Settings REST API

Configure a strategy when creating a combo via the API endpoint defined in src/app/api/settings/combo/route.ts:

POST /api/v1/settings/combo
{
  "name": "cost-aware-combo",
  "targets": [
    { "providerId": "openai", "modelId": "gpt-4o-mini" },
    { "providerId": "anthropic", "modelId": "claude-3-sonnet-20240229" }
  ],
  "strategy": "cost-optimized"
}

The strategy field is validated against ROUTING_STRATEGY_VALUES by the Zod schema in src/shared/validation/schemas/combo.ts.

Via Combo JSON Files

For CLI-based workflows, save a JSON file to ~/.omniroute/combo/my-combo.json:

{
  "name": "reliable-fallback",
  "targets": [
    { "providerId": "anthropic", "modelId": "claude-3-opus-20240229" },
    { "providerId": "openai", "modelId": "gpt-4o" }
  ],
  "strategy": "reset-aware"
}

Import using omniroute combo import ~/.omniroute/combo/my-combo.json.

Via MCP Tools

Change strategies at runtime using the MCP server tool defined in open-sse/mcp-server/tools/advancedTools.ts:

omniroute mcp --tool set_routing_strategy --args '{
  "combo": "my-combo",
  "strategy": "headroom"
}'

Programmatic SDK Usage

When using the OmniRoute SDK, the TypeScript compiler validates the strategy at build time:

import { OmniRouteClient } from '@omniroute/sdk'

const client = new OmniRouteClient({ apiKey: process.env.OMNIROUTE_API_KEY })

await client.createCombo({
  name: 'context-heavy',
  targets: [
    { providerId: 'openai', modelId: 'gpt-4o-128k' },
    { providerId: 'anthropic', modelId: 'claude-3-opus-20240229' }
  ],
  strategy: 'context-optimized'
})

Validation and Type Safety

The system enforces strategy validity at multiple layers. The normalizeRoutingStrategy() function ensures unknown values fall back to "priority", while the Zod schema in src/shared/validation/schemas/combo.ts rejects API requests containing invalid strategies. TypeScript types (RoutingStrategyValue) provide compile-time safety for SDK users.

The actual dispatch logic resides in open-sse/services/combo/comboSetup.ts, which interprets the combo's strategy field and coordinates with rateLimitManager.ts and usage.ts to apply the selection algorithm in real-time.

Summary

• Failover: Use priority for primary/backup setups; fill-first for quota exhaustion workflows.
• Load Distribution: Use weighted for proportional traffic splitting; round-robin for equal rotation; p2c or least-used for dynamic load balancing.
• Reliability: Use reset-aware or reset-window to avoid recently restarted providers; lkgp for session stickiness.
• Cost & Resources: Use cost-optimized for budget control; headroom and context-optimized for large token payloads.
• Advanced: Use auto for delegated optimization; fusion for parallel multi-provider execution.

Frequently Asked Questions

What is the difference between random and strict-random?

Both select targets randomly, but strict-random commits to the initially chosen provider even if it fails during processing, whereas random allows the system to retry on a different provider if the first attempt errors. Use strict-random only when you require request isolation for debugging or compliance reasons.

Can I change routing strategies without restarting OmniRoute?

Yes. Strategies can be updated at runtime via the MCP tool set_routing_strategy or by calling the Settings REST API (PUT /api/v1/settings/combo/{name}). Changes are persisted to the database through src/lib/db/combo.ts and take effect immediately for new requests.

What happens if I specify a strategy not in the official list?

The normalizeRoutingStrategy() function in src/shared/constants/routingStrategies.ts automatically falls back to "priority" if an invalid string is provided, ensuring the system remains operational even with malformed configuration.

When should I use the fusion strategy?

Use fusion when maximum reliability or lowest possible latency is required, as it sends requests to multiple providers simultaneously and returns the first successful response (or aggregates them). Note that this incurs higher costs due to duplicate API calls and is marked as experimental in the current codebase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}