How the Cost-Optimized Routing Strategy Works in OmniRoute

The cost-optimized routing strategy automatically sorts AI model candidates by input token price, selecting the cheapest available provider first while preserving fallback chains for resilience.

OmniRoute is an open-source AI gateway that intelligently routes requests across multiple LLM providers. The cost-optimized routing strategy enables organizations to minimize inference expenses by automatically prioritizing the cheapest available models without sacrificing reliability, as implemented in the diegosouzapw/OmniRoute repository.

Core Architecture of the Cost-Optimized Strategy

The strategy operates at two distinct layers within the OmniRoute codebase: combo target ordering for multi-model routing and connection-level selection for direct provider fallback.

Combo Target Ordering

When a request dispatches a combo configuration with strategy: "cost-optimized", the system invokes sortTargetsByCost() in open-sse/services/combo/targetSorters.ts. This function delegates to sortModelsByCost() (lines 61-89), which re-orders the resolved target list so that models with the lowest input price appear first. The sorting logic retrieves pricing data via getPricingForModel() from the lazily imported src/lib/localDb module, ensuring the strategy always reflects current provider catalog data.

Connection-Level Fallback

For single-provider requests that specify fallbackStrategy: "cost-optimized" outside of combo contexts, the logic resides in src/sse/services/auth.ts (lines 76-82). The getProviderCredentials() function sorts available connections by their priority field—where lower numeric values indicate cheaper or preferred connections—and selects the first candidate. This mirrors the combo behavior but operates on connection objects rather than model identifiers.

Step-by-Step Execution Flow

The complete cost-optimized routing flow follows six deterministic steps:

1. Resolve candidate models – The combo builder aggregates ResolvedComboTarget objects representing concrete provider/model pairs.

2. Lookup pricing – sortModelsByCost() queries getPricingForModel(provider, model) for each candidate. If pricing metadata is unavailable, the model receives an infinite cost value, ensuring it sorts to the end of the list.

3. Sort by cost – Candidates are sorted ascending by numeric input price using the comparison a.cost - b.cost. The cheapest model occupies the first position.

4. Tie-break handling – When multiple models share identical pricing, the sort preserves original array order (stable sort). This deterministic behavior is verified by unit tests such as "cost-optimized preserves the original order on price ties".

5. Connection fallback – If the selected provider fails during the request cycle, combo fallback logic attempts the next cheapest target, maintaining cost ordering throughout retries.

6. Apply strategy ordering – The generic applyStrategyOrdering() function in open-sse/services/combo/applyStrategyOrdering.ts (lines 53-94) dispatches to sortTargetsByCost() when the strategy name matches "cost-optimized", logs the selected model, and applies any manifest-routing hints.

Implementation Details and Key Functions

The strategy integrates with OmniRoute's broader routing system through several critical files:

• open-sse/services/combo/targetSorters.ts – Implements sortTargetsByCost() and sortModelsByCost(), containing the core price-comparison logic.

• open-sse/services/combo/applyStrategyOrdering.ts – Central dispatch that branches on strategy names and orchestrates the ordering application.

• src/sse/services/auth.ts – Handles cost-optimized connection selection for non-combo requests via priority-based sorting.

• src/shared/constants/routingStrategies.ts (lines 45-52) – Declares the strategy enumeration ROUTING_STRATEGY_VALUES and UI metadata (icon: "savings"), exposing "cost-optimized" as a public configuration option.

• src/lib/localDb.ts – Provides the getPricingForModel() function used to retrieve per-model input costs from the centralized provider catalog.

Code Examples

The following examples demonstrate configuring cost-optimized routing in both combo and direct fallback scenarios:

// Example: Creating a combo that uses the cost-optimized strategy
const combo = {
  id: "combo-001",
  name: "Cost Saver",
  strategy: "cost-optimized",   // Tells the router to sort by price
  enabled: true,
  targets: [
    "openai/gpt-4o-mini",      // $0.15 /M input tokens
    "google/gemini-2.5-pro",   // $2.00 /M input tokens
  ],
};

// Dispatch a chat request – the router automatically picks the cheapest model
await fetch("/api/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "auto/cost-optimized", // Combo resolver expands this shorthand
    messages: [{ role: "user", content: "Hello!" }],
  }),
});

// Direct use of the cost-optimized fallback strategy (no combo)
await fetch("/api/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-4o-mini",
    fallbackStrategy: "cost-optimized", // Selects connection with lowest priority
    messages: [{ role: "user", content: "How cheap can I go?" }],
  }),
});

Resilience and Fallback Behavior

The cost-optimized strategy participates fully in OmniRoute's resilience mechanisms. Because the sorting functions are pure and deterministic, they satisfy the isDeterministicStrategy("cost-optimized") checks required for caching. If the cheapest provider is temporarily unavailable due to circuit-breaker states or connection cooldowns, the system automatically attempts the next-cheapest target in the sorted list. This ensures that cost optimization never compromises availability, as the strategy maintains the complete ordered chain of candidates throughout the request's retry cycle.

Summary

• The cost-optimized routing strategy sorts AI model candidates by input token pricing, placing the cheapest option first.
• Implementation spans targetSorters.ts for combo ordering and auth.ts for connection-level fallback.
• Missing pricing data defaults to infinite cost, pushing unpriced models to the end of the candidate list.
• A stable sort preserves original ordering when prices are identical, ensuring deterministic behavior.
• The strategy integrates with circuit-breakers and cooldown mechanisms, automatically falling back to the next-cheapest provider if the primary fails.

Frequently Asked Questions

How does the cost-optimized strategy handle models with identical pricing?

When two or more models share the exact same input price, the sorting algorithm preserves their original array order because sortModelsByCost() implements a stable sort. This deterministic tie-breaking prevents erratic routing behavior and is explicitly verified by unit tests in the OmniRoute test suite.

What happens when pricing data is unavailable for a model?

If getPricingForModel() cannot locate pricing metadata for a specific provider/model combination, the function assigns an infinite cost value to that candidate. This ensures models with missing pricing data sort to the end of the list, making them available as last-resort fallbacks rather than being excluded entirely.

Can cost-optimized routing be used outside of combo configurations?

Yes. While the primary implementation handles combo targets via applyStrategyOrdering(), you can enable cost-optimized selection for direct provider requests by setting fallbackStrategy: "cost-optimized". In this mode, getProviderCredentials() in src/sse/services/auth.ts sorts available connections by their priority field and selects the lowest-numbered (cheapest) connection.

How does the strategy maintain availability if the cheapest provider fails?

The strategy maintains the complete sorted list of candidates throughout the request lifecycle. If the first (cheapest) provider fails due to network errors, rate limits, or circuit-breaker states, OmniRoute's fallback logic automatically attempts the second-cheapest target, continuing down the cost-ordered chain until a successful response is obtained or all candidates are exhausted.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}