OmniRoute Routing Strategies: Dynamic Model Selection Explained

Yes, OmniRoute supports multiple routing strategies—including priority, weighted, round-robin, auto-combo, and task-aware—which the Combo engine evaluates dynamically at runtime to select the optimal upstream model for every request.

OmniRoute is an open-source AI model router that implements a sophisticated Combo engine for intelligent request distribution. According to the diegosouzapw/OmniRoute source code, the system supports various OmniRoute routing strategies that can be configured per combo and evaluated on-the-fly using live telemetry such as latency stats, quota usage, and circuit-breaker state.

How Routing Strategies Work in OmniRoute

The routing decision is re-computed for each request inside the combo service (open-sse/services/combo.ts), allowing instant adaptation to provider health changes and cost-optimization policies. When a request arrives specifying a combo model, the engine invokes applyStrategyOrdering from open-sse/services/combo/applyStrategyOrdering.ts to determine the target upstream based on the configured strategy.

The available strategies are defined in src/shared/constants/routingStrategies.ts and include:

• Priority – Strict ordering based on preference lists
• Weighted – Probabilistic distribution across targets
• Round-robin – Sequential rotation through available models
• Auto-combo – Intelligent selection using latency, cost, and quota metrics
• Task-aware – Routing based on specific task requirements and context affinity

Dynamic Strategy Selection with Live Telemetry

The auto-combo strategy implements the most sophisticated OmniRoute routing strategy, leveraging real-time data to make optimal selections. The buildAutoCandidates function (located at line 93 in open-sse/services/combo.ts) gathers live telemetry including:

• getModelLatencyStats – Current response time metrics
• getPricingForModel – Cost per token data
• getCircuitBreaker – Provider health status
• Session stickiness and context-cache pins via isPinnedModelDurablyUnhealthy

This data feeds into the strategy evaluation pipeline, where unhealthy or exhausted providers are automatically deprioritized without requiring configuration changes.

Wildcard Expansion and Runtime Target Resolution

OmniRoute routing strategies operate on dynamically resolved target lists through wildcard expansion. The expandProviderWildcardsInCombo function (in open-sse/services/combo/providerWildcard.ts) converts entries like openai/* or anthropic/claude-* into concrete model targets at request time.

The resolveComboTargets function then combines these expanded wildcards with the selected strategy to generate the final routing decision. This architecture means that adding new providers or models requires no code changes—the next request automatically includes new targets that match existing wildcard patterns.

Configuring Routing Strategies

Strategies are assigned when creating or updating combos via the database module at src/lib/db/combos.ts. Changes take effect immediately without server restarts.

Creating a Combo with the Auto Strategy

import { createCombo } from "@/src/lib/db/combos";

await createCombo({
  name: "intelligent-router",
  models: [
    "openai/*",
    "anthropic/claude-*",
    "groq/*"
  ],
  strategy: "auto",  // Enables telemetry-driven selection
  config: {
    maxRetries: 2,
    retryDelayMs: 1000,
  },
});

Priority-Based Strategy Configuration

await createCombo({
  name: "priority-fallback",
  models: [
    "openai/gpt-4",
    "anthropic/claude-3-opus",
    "openai/gpt-3.5-turbo"
  ],
  strategy: "priority",  // Tries models in strict order
});

Updating Strategies at Runtime

import { updateCombo } from "@/src/lib/db/combos";

// Switch from priority to weighted strategy instantly
await updateCombo("priority-fallback", {
  strategy: "weighted",
  weights: [0.7, 0.2, 0.1],  // 70% GPT-4, 20% Claude, 10% GPT-3.5
});

Strategy Override and Request Routing

When making API requests, you trigger the routing strategy by referencing the combo name:

curl -X POST https://router.example.com/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "combo:intelligent-router",
        "messages": [{"role":"user","content":"Explain quantum tunneling"}]
      }'

The route handler (src/app/api/v1/chat/completions/route.ts) forwards this to handleComboChat, which invokes the strategy pipeline. The applyStrategyOrdering function then executes the specific logic for the combo's configured strategy—whether that's calculating weighted probabilities, rotating through a round-robin queue, or scoring auto-combo candidates based on current latency and quota availability.

Summary

• OmniRoute routing strategies include priority, weighted, round-robin, auto-combo, and task-aware, defined in src/shared/constants/routingStrategies.ts.
• The auto-combo strategy uses live telemetry via buildAutoCandidates and functions like getModelLatencyStats to make intelligent routing decisions.
• Wildcard expansion (expandProviderWildcardsInCombo) enables dynamic target lists that update automatically as providers change.
• Strategies are configured per combo in src/lib/db/combos.ts and can be updated at runtime without restarting the server.
• Every request triggers fresh strategy evaluation through applyStrategyOrdering, ensuring immediate adaptation to provider health and quota changes.

Frequently Asked Questions

How do I change routing strategies without restarting OmniRoute?

You update the combo configuration using the database module. The updateCombo function in src/lib/db/combos.ts modifies the SQLite storage, and because resolveComboTargets recomputes the strategy for each request, changes take effect on the very next API call. No server restart is required.

Which OmniRoute routing strategy should I use for high availability?

Use the priority strategy with health-aware fallbacks or the auto strategy. The auto-combo strategy automatically deprioritizes unhealthy providers through isPinnedModelDurablyUnhealthy checks and circuit-breaker state evaluation, ensuring requests route to available models even during outages.

Can I combine multiple routing strategies in one combo?

Combos use a single primary strategy, but the auto-combo strategy effectively combines multiple decision factors—latency, cost, quota, and task requirements—into a unified scoring mechanism. For explicit multi-strategy behavior, create separate combos and route between them at the application layer.

Does OmniRoute support weighted load balancing across providers?

Yes. The weighted strategy allows probabilistic distribution across multiple upstream models by assigning weights in the combo configuration. The applyStrategyOrdering function processes these weights to distribute traffic according to your specified ratios, such as 70% to one provider and 30% to another.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}