How to Use the Priority Combo Routing Strategy in OmniRoute

The priority combo routing strategy in OmniRoute routes requests sequentially through a provider priority list, returning the first successful response while halting immediately on quota exhaustion errors to prevent wasteful fallback attempts.

OmniRoute is an open-source AI routing proxy that supports 19 distinct combo routing strategies for intelligent multi-provider failover. The priority combo routing strategy serves as the default mechanism, offering deterministic provider selection based on hierarchical precedence and model-level weights defined in the provider registry.

How the Priority Strategy Works

Target List Resolution

When a request enters the combo routing pipeline, the resolveComboTargets() function in open-sse/services/combo.ts constructs the candidate provider list. The system orders targets by provider-level priority defined in src/shared/constants/providers.ts and model-level weight specified in open-sse/config/providerRegistry.ts. This creates a deterministic hierarchy where preferred providers appear first in the execution sequence.

Sequential Execution and Validation

The strategy traverses the ordered list sequentially, calling handleSingleModel() for each target. This function performs three critical validations before dispatch:

1. Request validation against the target model schema
2. Circuit-breaker state checks to skip providers with open breakers or active connection cooldowns
3. Executor selection from open-sse/executors/* based on the provider type

If validation passes, the request dispatches through the appropriate executor. If the target returns a transient failure (5xx errors, timeout), the system logs the failure in open-sse/services/accountFallback.ts and immediately proceeds to the next lower-priority target.

Success Criteria and Early Termination

The priority strategy implements a first-success-wins policy. Upon receiving an HTTP 2xx response with a valid payload from any target, the routing loop terminates immediately. The successful response streams back to the client through open-sse/transformer/responseTransformer.ts (or responsesTransformer.ts for Responses API requests) without attempting remaining lower-priority providers. This minimizes latency by avoiding redundant upstream calls.

Quota Exhaustion Cutoff

Unlike transient failures, quota exhaustion triggers a hard stop. When a target returns a 429 status code indicating quota exhaustion, the strategy halts the entire combo request immediately. This quota exhaustion cutoff, verified in tests/unit/combo-priority-quota-exhaustion-cutoff-5923.test.ts, prevents the system from consuming credits at lower-priority providers when the primary account limits are reached, ensuring compliance with usage policies and preventing unnecessary costs.

Usage Examples

Default Priority Routing

When you omit the strategy field, OmniRoute automatically applies the priority strategy:

import { fetchChat } from '@omniroute/open-sse';

const body = {
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Explain priority routing.' }]
  // strategy defaults to 'priority'
};

const response = await fetchChat('/v1/chat/completions', { 
  method: 'POST', 
  body 
});

This configuration routes the request to the highest-priority provider hosting gpt-4. If that provider is unavailable due to circuit breaker or cooldown states, the system automatically fails over to the next provider in the hierarchy.

Explicit Strategy Declaration

To explicitly configure the strategy or combine it with specific target arrays:

const body = {
  model: 'claude-2',
  messages: [{ role: 'user', content: 'How does priority work?' }],
  strategy: 'priority',
  targets: [
    { provider: 'anthropic', model: 'claude-2' },
    { provider: 'aws-bedrock', model: 'anthropic.claude-v2' }
  ]
};

const resp = await fetchChat('/v1/chat/completions', { 
  method: 'POST', 
  body 
});

Even when specifying explicit targets, the priority strategy respects the order provided in the targets array, attempting each sequentially until one succeeds.

Handling Quota Exhaustion

Implement error handling to catch quota exhaustion scenarios where the strategy aborts without trying fallback providers:

try {
  const resp = await fetchChat('/v1/chat/completions', {
    method: 'POST',
    body: {
      model: 'gpt-4',
      messages: [{ role: 'user', content: 'Generate content' }],
      strategy: 'priority'
    }
  });
  const data = await resp.json();
} catch (err) {
  // Catches quota exhaustion (429) from highest-priority provider
  // Lower-priority providers are NOT attempted
  console.error('Primary provider quota exhausted:', err);
}

Key Source Files

The priority strategy implementation spans several critical components:

• open-sse/services/combo.ts – Contains the core routing loop, resolveComboTargets(), and handleSingleModel() execution logic.
• open-sse/services/accountFallback.ts – Manages circuit-breaker states, connection cooldowns, and provider availability checks.
• open-sse/config/providerRegistry.ts – Defines model-level weights that influence target ordering.
• src/shared/constants/providers.ts – Specifies the base provider priority hierarchy.
• open-sse/transformer/responseTransformer.ts and open-sse/transformer/responsesTransformer.ts – Handle response formatting and streaming.
• tests/unit/combo-priority-quota-exhaustion-cutoff-5923.test.ts – Validates the quota exhaustion cutoff behavior.
• tests/unit/8370-priority-affinity-reorder.test.ts – Ensures correct priority ordering with provider affinity logic.
• docs/routing/AUTO-COMBO.md – Documents all 19 combo strategies including priority algorithm details.

Summary

• The priority combo routing strategy is OmniRoute's default mechanism for deterministic multi-provider failover.
• Targets execute sequentially based on provider priority and model weights until the first successful response.
• Transient failures (5xx, timeouts) trigger automatic failover to the next target, while quota exhaustion errors (429) halt execution immediately to prevent credit waste.
• The strategy minimizes latency by avoiding parallel requests and redundant provider calls.
• Configuration requires no explicit strategy declaration for default behavior, or set strategy: 'priority' for explicit selection.

Frequently Asked Questions

How does OmniRoute determine provider priority order?

OmniRoute constructs the execution list by combining provider-level priority constants from src/shared/constants/providers.ts with model-level weights defined in open-sse/config/providerRegistry.ts. The resolveComboTargets() function sorts candidates using these values, ensuring higher-priority providers are attempted first. Provider affinity rules tested in 8370-priority-affinity-reorder.test.ts may further adjust ordering based on recent successful connections.

What happens if the highest-priority provider is temporarily unavailable?

If handleSingleModel() encounters a circuit breaker in an open state or a connection under cooldown (managed in open-sse/services/accountFallback.ts), the priority strategy skips that provider and immediately attempts the next target in the list. This failover happens automatically without client intervention, provided lower-priority providers exist in the resolved target list.

Why does the priority strategy stop on quota exhaustion instead of falling back?

The quota exhaustion cutoff is a deliberate cost-control mechanism. When a provider returns a 429 quota-exhausted error, continuing to lower-priority providers would consume credits elsewhere while potentially violating the original provider's rate limits. The strategy aborts immediately, as verified by combo-priority-quota-exhaustion-cutoff-5923.test.ts, ensuring predictable billing and preventing wasteful requests to backup providers when primary limits are reached.

Can I use the priority strategy with explicit provider targeting?

Yes. When you supply a targets array in the request body, the priority strategy respects the explicit order of that array, attempting each specified provider/model combination sequentially. This allows fine-grained control over failover chains while maintaining the strategy's deterministic, first-success behavior and quota protection logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}