How OmniRoute's Cost-Optimized Routing Strategy Minimizes AI Spending

The cost-optimized strategy sorts eligible provider/model candidates by their per-token input price and routes requests to the cheapest option first, automatically falling back to the next-cheapest on failure.

OmniRoute's cost-optimized combo routing strategy is one of 19 available algorithms in the open-source AI gateway. It lets developers minimize inference costs without sacrificing reliability by combining dynamic price-based sorting with automatic failover. This article explains exactly how the strategy works, from strategy registration through the dispatch loop, using source code from diegosouzapw/OmniRoute.

What Is the Cost-Optimized Routing Strategy?

The cost-optimized strategy belongs to OmniRoute's combo routing system, which allows a single model parameter to expand into multiple provider/model candidates. Instead of hardcoding a provider, you specify combo:cost-optimized and let the gateway choose based on real-time pricing data.

When enabled, the strategy:

1. Generates all healthy candidates for a request
2. Looks up each candidate's input price from the provider catalog
3. Sorts candidates from cheapest to most expensive
4. Attempts the cheapest candidate first
5. Falls back to the next-cheapest if the first fails

How the Strategy Is Registered

The strategy name is defined in the routing-strategy enumeration at src/shared/constants/routingStrategies.ts:

// Lines 10-16
export enum RoutingStrategy {
  // ... other strategies
  COST_OPTIMIZED = 'cost-optimized',
  // ...
}

This registration makes cost-optimized available as a valid strategy value in combo manifests and API requests.

Candidate Generation and Health Filtering

When a request arrives with model: "combo:cost-optimized", the combo engine in open-sse/services/combo.ts builds a list of candidates. Each candidate represents a concrete provider connection plus the specific model to invoke.

Before price sorting occurs, candidates pass through standard health checks:

• Circuit-breaker status — providers with open circuits are excluded
• Cooldown periods — recently failed providers are temporarily skipped
• Model lockout — models marked unavailable are filtered out

Only healthy candidates proceed to price lookup.

Price Lookup from the Provider Catalog

For each surviving candidate, the engine consults the provider catalog loaded at startup from the provider registry. The catalog contains input price values (e.g., $0.15/M for gpt-4o-mini, $5.00/M for claude-3-opus).

The price is attached to the candidate object as candidate.price:

// Conceptual flow from combo.ts
const candidate = {
  provider: 'openai',
  model: 'gpt-4o-mini',
  price: 0.15, // $ per million input tokens
  // ... connection details
};

If pricing data is missing for a candidate, the engine logs a skip message and may exclude it from consideration.

The Sorting Algorithm: Cheapest First

The core logic resides in the applyStrategyOrdering helper. For cost-optimized specifically, candidates are sorted ascending by price:

// From applyStrategyOrdering implementation
case RoutingStrategy.COST_OPTIMIZED:
  candidates.sort((a, b) => a.price - b.price);
  break;

This simple numeric sort places the cheapest candidate at index 0, the second-cheapest at index 1, and so on. The ordering is verified in tests/unit/auto-combo-codex-responses-3509.test.ts (lines 30-34) and tests/unit/combo-routing-engine.test.ts (lines 1812-1826).

The Dispatch Loop and Automatic Fallback

Once ordered, the combo engine executes a sequential dispatch loop:

// Simplified from combo.ts dispatch logic
for (const candidate of orderedCandidates) {
  try {
    const response = await dispatchToProvider(candidate, request);
    return response; // First success wins
  } catch (error) {
    logFailure(candidate, error);
    continue; // Try next-cheapest candidate
  }
}

The first successful response returns immediately to the client. If the cheapest candidate fails—due to provider error, rate limit, circuit-breaker open, or timeout—the engine automatically retries with the next-cheapest candidate.

This fallback behavior is explicitly tested in tests/unit/combo-strategy-fallbacks.test.ts (lines 202-214) with the assertion: "cost-optimized falls back to the next-cheapest target when the cheapest fails".

Handling Price Ties Deterministically

When two candidates have identical pricing, OmniRoute preserves the original order from the combo manifest. This guarantees deterministic routing—important for reproducible behavior and testing.

The test at tests/unit/combo-strategy-fallbacks.test.ts (lines 243-255) verifies this: "cost-optimized preserves the original order on price ties".

Structured Logging for Debugging

Throughout execution, the engine emits structured logs prefixed with [cost-optimized]:

[cost-optimized] Evaluating 5 candidates
[cost-optimized] Selected provider: openai/gpt-4o-mini @ $0.15/M (rank 1)
[cost-optimized] Provider openai/gpt-4o-mini failed with 429, trying next
[cost-optimized] Fallback to azure/gpt-4o-mini @ $0.16/M (rank 2)

These logs appear in the live smoke tests at tests/integration/combo-live/cost-and-fusion.live.test.ts (lines 206-222), enabling operators to trace routing decisions in production.

Using Cost-Optimized in Practice

API Request Example

import fetch from "node-fetch";

const response = await fetch("http://localhost:20128/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "combo:cost-optimized",
    messages: [{ role: "user", content: "Explain quantum computing" }],
  }),
});

const result = await response.json();

The combo:cost-optimized prefix activates the strategy. You can combine it with other parameters like fallbackStrategy or maxRetries per request.

Inspecting Candidate Ordering (Debug Mode)

import { resolveComboCandidates } from "@/open-sse/services/combo";

const request = {
  model: "combo:cost-optimized",
  messages: [{ role: "user", content: "Hello" }]
};

const candidates = await resolveComboCandidates(request);
// Apply same sorting as cost-optimized strategy
candidates.sort((a, b) => a.price - b.price);

console.log(candidates.map(c => 
  `${c.provider}/${c.model} @ $${c.price}/M`
));
// Output: [ 'openai/gpt-4o-mini @ $0.15/M', 'azure/gpt-4o-mini @ $0.16/M', ... ]

Key Source Files

File	Purpose
src/shared/constants/routingStrategies.ts	Strategy enumeration (lines 10-16)
open-sse/services/combo.ts	Core combo engine with applyStrategyOrdering
tests/unit/combo-strategy-fallbacks.test.ts	Fallback behavior and tie-handling tests (lines 202-214, 243-255)
tests/unit/combo-routing-engine.test.ts	Price-sorting verification (lines 1812-1826)
tests/integration/combo-live/cost-and-fusion.live.test.ts	Live smoke tests with structured logging (lines 206-222)

Summary

• Cost-optimized is one of 19 combo routing strategies in OmniRoute v3.8.51
• It sorts healthy candidates by ascending input price using catalog data
• The cheapest candidate is tried first, with automatic fallback to next-cheapest on failure
• Deterministic ordering is preserved for equal-price ties
• Structured logging with [cost-optimized] prefix aids production debugging
• All behavior is covered by unit and integration tests in the repository

Frequently Asked Questions

What happens if no pricing data exists for a candidate?

Candidates missing price information are logged with a skip message and typically excluded from the sorted list. The engine proceeds with candidates that have valid catalog pricing.

Does cost-optimized consider output token prices?

No—according to the source code, only input price (candidate.price) is used for sorting. Output pricing is not factored into the cost-optimized ordering.

Can I combine cost-optimized with other selection criteria?

Yes. The cost-optimized strategy applies after health filters (circuit-breaker, cooldown, lockout) but within a combo definition, you can pair it with model constraints, region preferences, or custom manifests. The strategy ordering is the final step before dispatch.

How does cost-optimized differ from latency-optimized or quality-optimized strategies?

Latency-optimized sorts by estimated response time, quality-optimized by model capability scores, and cost-optimized strictly by monetary input price. All three use the same applyStrategyOrdering helper with different sort keys, sharing identical fallback and logging infrastructure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}