How the Priority Combo Routing Strategy Works in OmniRoute

The priority combo routing strategy processes targets sequentially by precedence and returns the first successful response, halting immediately on quota exhaustion errors.

OmniRoute is an open-source AI gateway that supports 19 built-in combo routing strategies for distributing requests across multiple LLM providers. The priority strategy serves as the default routing method when no explicit strategy is specified, offering deterministic failover by attempting providers in a strict order of preference until one succeeds.

What Is the Priority Combo Routing Strategy?

The priority strategy implements a first-successful-target approach. When a request arrives, OmniRoute builds an ordered list of candidate targets based on provider-level priorities defined in src/shared/constants/providers.ts and model-level weights from open-sse/config/providerRegistry.ts. The system then executes these targets sequentially, returning immediately upon receiving a valid response while respecting circuit-breaker states and quota limits.

Step-by-Step Execution Flow

Target List Creation

OmniRoute initiates the routing process by calling resolveComboTargets() within open-sse/services/combo.ts. This function constructs the candidate list by evaluating:

• The requested model and available providers
• Provider-level priority constants from the configuration
• Model-level weights defined in the provider registry
• Explicit targets arrays supplied in the client request

The resulting array is sorted by precedence, ensuring higher-priority providers appear first in the execution queue.

Sequential Execution

Once the target list is established, the strategy enters a sequential traversal loop. For each target, OmniRoute invokes handleSingleModel(), which performs the following validations:

1. Validates the request structure and parameters
2. Checks the provider-level circuit-breaker state
3. Verifies connection cooldown status via accountFallback.ts
4. Dispatches the request through the appropriate executor in open-sse/executors/

Early Return on Success

The priority strategy implements an early-exit optimization. As soon as a target returns an HTTP 2xx status with a valid payload, the combo routing loop terminates immediately. The successful response streams back to the client through open-sse/transformer/responseTransformer.ts (or responsesTransformer.ts for Responses API calls) without attempting any remaining lower-priority targets.

Failure and Fallback Handling

When a target fails due to transient conditions—such as an open circuit breaker, connection cooldown, or upstream 5xx error—OmniRoute logs the failure in open-sse/services/accountFallback.ts and marks the connection as temporarily unavailable. The strategy then proceeds to the next target in the priority list, ensuring graceful degradation without exposing provider-specific errors to the client.

Quota Exhaustion Cutoff

Unlike other failure modes, quota exhaustion triggers an immediate halt. If a target returns a 429 error with a quota-exhausted signal, the priority strategy stops the entire combo routing process. This behavior, verified in tests/unit/combo-priority-quota-exhaustion-cutoff-5923.test.ts, prevents wasting credits on lower-priority providers when usage limits are reached, enforcing strict adherence to quota policies.

Key Implementation Files

Understanding the priority strategy requires familiarity with these core components:

• open-sse/services/combo.ts – Contains the main combo routing engine, including resolveComboTargets() and the priority loop that selects the first successful target
• open-sse/services/accountFallback.ts – Manages circuit-breaker logic, connection cooldowns, and model lockout states that determine target availability
• open-sse/transformer/responseTransformer.ts – Handles response formatting and streaming for standard chat completions
• open-sse/transformer/responsesTransformer.ts – Processes responses for the Responses API format
• tests/unit/combo-priority-quota-exhaustion-cutoff-5923.test.ts – Unit test validating the quota exhaustion cutoff behavior
• tests/unit/8370-priority-affinity-reorder.test.ts – Ensures priority ordering respects provider affinity and reordering logic

Practical Code Examples

Basic Usage with Default Priority

When you omit the strategy field, OmniRoute automatically applies the priority strategy:

import { fetchChat } from '@omniroute/open-sse';

const body = {
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Explain priority routing.' }],
  // No strategy specified - defaults to priority
};

const response = await fetchChat('/v1/chat/completions', { 
  method: 'POST', 
  body 
});
console.log(await response.json());

This request routes to the highest-priority provider hosting gpt-4. If that provider is temporarily unavailable due to circuit breaker or cooldown, the request automatically falls back to the next provider in the priority list.

Explicit Priority Strategy Selection

You can explicitly declare the strategy to ensure predictable behavior when supplying multiple targets:

const body = {
  model: 'claude-2',
  messages: [{ role: 'user', content: 'How does priority work?' }],
  strategy: 'priority',  // Explicitly select priority routing
  targets: [
    { provider: 'anthropic', model: 'claude-2' },
    { provider: 'aws-bedrock', model: 'claude-2' }
  ]
};

const resp = await fetchChat('/v1/chat/completions', { 
  method: 'POST', 
  body 
});

Even with multiple targets defined, the strategy processes them in order and returns the first successful response.

Handling Quota Exhaustion

The priority strategy distinguishes between transient failures and quota limits:

try {
  const resp = await fetchChat('/v1/chat/completions', {
    method: 'POST',
    body: {
      model: 'gpt-4',
      messages: [{ role: 'user', content: 'What is the limit?' }],
      strategy: 'priority',
    },
  });
  console.log(await resp.json());
} catch (err) {
  // Catches 429 quota exhaustion from highest-priority provider
  // Lower-priority providers are NOT attempted
  console.error('Quota exhausted – request aborted:', err);
}

When the highest-priority provider returns a quota-exhausted error, the strategy aborts immediately rather than consuming credits on backup providers.

Summary

• The priority combo routing strategy processes provider targets sequentially based on configured precedence levels
• It returns the first successful response immediately, ensuring low latency by avoiding unnecessary upstream calls
• Transient failures (5xx errors, circuit breakers, cooldowns) trigger automatic fallback to the next priority target
• Quota exhaustion (429 errors) halts the entire routing process to prevent credit waste on lower-priority providers
• The strategy serves as the default routing method when no explicit strategy is specified in the request
• Implementation spans open-sse/services/combo.ts, accountFallback.ts, and the transformer modules

Frequently Asked Questions

What is the default routing strategy in OmniRoute?

When a request does not specify a strategy field, OmniRoute automatically applies the priority combo routing strategy. This default behavior ensures deterministic routing based on provider precedence defined in src/shared/constants/providers.ts without requiring explicit client configuration.

How does priority routing handle provider failures?

The strategy distinguishes between transient and terminal failures. For transient issues like HTTP 5xx errors, open circuit breakers, or connection cooldowns, OmniRoute marks the provider unavailable via accountFallback.ts and immediately attempts the next target in the priority list. This happens transparently without returning errors to the client unless all targets exhaust.

What happens when all providers return quota exhaustion errors?

If the highest-priority provider returns a 429 quota-exhausted error, the priority strategy aborts immediately without trying lower-priority providers. This quota exhaustion cutoff prevents wasting credits across multiple providers. The 429 error propagates directly to the client, signaling that usage limits have been reached for the requested model tier.

How is the priority order determined?

OmniRoute calculates target precedence using a combination of provider-level priorities from src/shared/constants/providers.ts and model-level weights from open-sse/config/providerRegistry.ts. The resolveComboTargets() function in combo.ts sorts the final target list, placing higher-priority providers first. Explicit targets arrays in client requests override default ordering while maintaining relative priority weights.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}