How the Cost-Optimized Routing Strategy Works in OmniRoute

The cost-optimized routing strategy sorts candidate provider-model pairs by their input price (cheapest first) and dispatches requests to the lowest-cost healthy provider, falling back to more expensive options only if the primary fails.

OmniRoute's cost-optimized routing strategy is a deterministic combo strategy designed to minimize API spending. When enabled, the system queries its pricing catalog, ranks all viable providers by their per-token input cost, and routes traffic to the cheapest available option. This article breaks down the implementation based on the OmniRoute source code in the diegosouzapw/OmniRoute repository.

Core Architecture

The cost-optimized strategy is implemented across three main components in the open-sse/services/combo/ directory.

Price-Based Sorting with sortModelsByCost

The foundation lives in open-sse/services/combo/targetSorters.ts (lines 66-85). This function:

1. Dynamically imports @/lib/db/settings to access the pricing database
2. Calls getPricingForModel(provider, model) for each candidate
3. Returns models sorted by ascending input price

// Conceptual flow from targetSorters.ts lines 66-85
const sortModelsByCost = async (models: string[]) => {
  const pricing = await import('@/lib/db/settings');
  const withCosts = await Promise.all(
    models.map(async (m) => {
      const cost = await pricing.getPricingForModel(m.provider, m.name);
      return { model: m, cost: cost?.input ?? Infinity };
    })
  );
  return withCosts.sort((a, b) => a.cost - b.cost).map(x => x.model);
};

Missing or failed pricing lookups are assigned Infinity, forcing those targets to the end of the list.

Target Resolution with sortTargetsByCost

Also in targetSorters.ts (lines 91-105), sortTargetsByCost maps the sorted model strings back to ResolvedComboTarget objects. Critically, this preserves the original relative order for ties—making the strategy deterministic.

Strategy Dispatch in applyStrategyOrdering

The open-sse/services/combo/applyStrategyOrdering.ts file (lines 53-55) contains the conditional branch that triggers cost sorting:

// From applyStrategyOrdering.ts lines 53-55
if (strategy === 'cost-optimized') {
  orderedTargets = await sortTargetsByCost(initialOrderedTargets);
}

Lines 93-95 add structured logging of the final ordering for observability.

Request Flow: Step by Step

Understanding the full pipeline helps debug routing decisions.

1. Request Intake

A client sends a chat completion request with the strategy flag:

POST /v1/chat/completions HTTP/1.1
Content-Type: application/json

{
  "model": "gpt-4o-mini",
  "messages": [{ "role": "user", "content": "Explain quantum tunneling." }],
  "comboStrategy": "cost-optimized"
}

2. Target Discovery

OmniRoute collects all healthy provider-model pairs matching gpt-4o-mini. Example candidates might include:

• openai/gpt-4o-mini at $0.15 per million input tokens
• gemini/gemini-1.5-flash at $0.35 per million
• azure/gpt-4o-mini at $0.18 per million

3. Cost Sorting Execution

The sortModelsByCost function resolves each provider's pricing, then sorts. The resulting order becomes: openai → azure → gemini.

4. Manifest Routing Filter (Optional)

If config.manifestRouting is enabled, generateRoutingHints may further restrict the list—filtering to premium-only models, for example. This happens after cost sorting, so the cheapest eligible model wins.

5. Dispatch with Fallback

The executor sends the request to orderedTargets[0]. On failure, it proceeds through the cost-sorted list, preserving cheap-first ordering for retries.

Edge Cases and Guarantees

Scenario	Behavior	Source Code Location
Missing pricing	Treated as Infinity, ranked last	targetSorters.ts implicit via ?? Infinity
Identical prices	Original order preserved (stable sort)	targetSorters.ts lines 91-105
All providers down	Returns standard error after exhausting list	applyStrategyOrdering.ts execution flow
Manifest conflicts	Cheapest manifest-eligible model selected	applyStrategyOrdering.ts post-sort filter

The strategy's determinism is formally declared in src/shared/constants/routingStrategies.ts (line 10), where "cost-optimized" is registered in DETERMINISTIC_STRATEGIES.

Programmatic Usage

For internal services consuming OmniRoute directly:

import { handleChat } from "@/open-sse/handlers/chatCore";

const response = await handleChat({
  body: {
    model: "auto",
    messages: [{ role: "user", content: "Summarize the latest AI paper." }],
    comboStrategy: "cost-optimized"
  }
});

console.log(response.result?.provider); // cheapest available
console.log(response.result?.cost);     // price per 1M tokens used

The handleChat function delegates to applyStrategyOrdering("cost-optimized", …), ensuring consistent behavior between API and library usage.

Test Coverage

The implementation is validated by:

• Unit test: cost-optimized selects the lowest priority account in tests/unit/sse-auth.test.ts (lines 1064-1065)
• Integration test: cost-optimized: cheapest model dispatched before expensive in tests/integration/combo-live/cost-and-fusion.live.test.ts (lines 165-173)

These tests confirm both the sorting logic in isolation and end-to-end behavior in a live-like environment.

Performance Characteristics

• Latency overhead: One pricing database query per unique (provider, model) pair; results are not cached within a single request
• Sorting complexity: O(n log n) where n is candidate count; typically n < 20
• Determinism: Guaranteed for identical input states due to stable sort and fixed pricing catalog

Summary

• The cost-optimized routing strategy minimizes spend by sorting providers by input price before dispatch
• Core logic resides in open-sse/services/combo/targetSorters.ts with entry point at applyStrategyOrdering.ts
• Missing prices default to Infinity; ties preserve original order for deterministic behavior
• The strategy integrates with manifest routing and provides cheap-first fallback resilience
• Enable via "comboStrategy": "cost-optimized" in API requests or direct handleChat calls

Frequently Asked Questions

What happens if two providers have exactly the same price?

Ties are resolved by preserving the original order of candidates. This stable sorting guarantee ensures deterministic routing—identical requests always produce identical provider orderings when prices match.

Does cost-optimized consider output token prices?

No. The current implementation sorts exclusively by input price (cost.input from the pricing catalog). Output pricing varies by usage patterns and is not factored into the routing decision.

Can I combine cost-optimized with other filters like region or latency?

Yes. The generateRoutingHints system (manifest routing) applies after cost sorting. You can restrict candidates by region, quality tier, or other attributes; the cheapest remaining eligible provider is then selected.

What if the cheapest provider is unhealthy or rate-limited?

OmniRoute's combo engine automatically proceeds to the next target in the cost-sorted list. The cheap-first ordering is preserved across retries, ensuring you only pay more when cheaper options genuinely fail.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}