How to Implement Cost-Optimized Routing Using OmniRoute's Live Catalog Pricing

OmniRoute automatically routes requests to the cheapest available AI model by applying the cost-optimized strategy, which sorts combo targets using live costPer1MTokens data from the provider registry.

Managing AI infrastructure costs requires dynamic routing based on real-time pricing. In the diegosouzapw/OmniRoute repository, you can implement cost-optimized routing using OmniRoute's live catalog pricing to ensure requests always hit the most economical provider first. This approach leverages the live-updated providerRegistry to sort targets by cost before dispatching requests.

How Cost-Optimized Routing Works

The cost-optimized strategy relies on four core components that fetch, sort, and dispatch requests based on live pricing data.

Core Architecture Components

Component	File Path	Role
Strategy Constants	src/shared/constants/routingStrategies.ts	Defines the "cost-optimized" identifier
Combo Definition	open-sse/services/combo.ts	Declares supported strategies including cost-optimized
Strategy Dispatcher	open-sse/services/combo/applyStrategyOrdering.ts	Routes to sortTargetsByCost when strategy matches
Target Sorter	open-sse/services/combo/targetSorters.ts	Implements sortTargetsByCost using costPer1MTokens
Provider Registry	open-sse/config/providerRegistry.ts	Stores live pricing catalog

The Sorting Algorithm

When a combo uses the cost-optimized strategy, applyStrategyOrdering in open-sse/services/combo/applyStrategyOrdering.ts executes the following flow:

1. Fetch Pricing: Retrieves costPer1MTokens (USD per million tokens) from the providerRegistry for each target model.
2. Build Targets: Creates ResolvedComboTarget objects inheriting the pricing field.
3. Sort: sortTargetsByCost sorts the array ascending by costPer1MTokens, preserving original order on ties.
4. Dispatch: The router iterates through the sorted list in handleComboChat, trying the cheapest provider first and falling back to more expensive options on failure.

Implementing Cost-Optimized Combos

Creating a Combo via the REST API

Define a combo with "strategy": "cost-optimized" to enable automatic cost sorting:

curl -X POST https://YOUR-OMNIRoute-INSTANCE/api/v1/combo \
  -H "Authorization: Bearer $OMNIRoute_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "name": "cheap-combo",
        "label": "Cheapest Model Combo",
        "strategy": "cost-optimized",
        "targets": [
          { "provider": "openai",   "model": "gpt-4o-mini" },
          { "provider": "anthropic","model": "claude-3-5-sonnet" },
          { "provider": "google",   "model": "gemini-1.5-flash" }
        ]
      }'

Invoking the Cost-Optimized Combo

Route requests through the combo name prefixed with combo::

curl -X POST https://YOUR-OMNIRoute-INSTANCE/api/v1/chat/completions \
  -H "Authorization: Bearer $OMNIRoute_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "combo:cheap-combo",
        "messages": [{ "role": "user", "content": "Explain quantum tunneling in plain English." }]
      }'

OmniRoute queries the live catalog and routes to the cheapest available target.

Migrating Existing Combos to Cost-Optimized

Update an existing combo's strategy without recreating it:

curl -X PATCH https://YOUR-OMNIRoute-INSTANCE/api/v1/combo/cheap-combo \
  -H "Authorization: Bearer $OMNIRoute_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "strategy": "cost-optimized" }'

CLI Configuration

Use the OmniRoute CLI to create cost-optimized combos interactively:

omniroute combo create \
  --name cheap-combo \
  --strategy cost-optimized \
  --target openai:gpt-4o-mini \
  --target anthropic:claude-3-5-sonnet \
  --target google:gemini-1.5-flash

Key Implementation Files

For custom modifications or debugging, inspect these source files:

• src/shared/constants/routingStrategies.ts: Contains the "cost-optimized" strategy constant.
• open-sse/services/combo/applyStrategyOrdering.ts: Dispatches to cost sorting logic at line 31 and handles execution flow at line 131.
• open-sse/services/combo/targetSorters.ts: Houses the sortTargetsByCost implementation at line 62.
• open-sse/config/providerRegistry.ts: Manages the live pricing catalog accessed via getRegistryEntry.

Summary

• Cost-optimized routing in OmniRoute uses the "cost-optimized" strategy identifier to sort targets by live pricing.
• The system pulls costPer1MTokens from providerRegistry.ts and sorts via sortTargetsByCost in targetSorters.ts.
• Configure combos via REST API or CLI by setting "strategy": "cost-optimized".
• The router tries the cheapest provider first, falling back to more expensive options only if necessary.
• Pricing data updates periodically from feeds like LiteLLM, ensuring routes reflect current market rates.

Frequently Asked Questions

What happens if two models have identical pricing?

When costPer1MTokens values are equal, sortTargetsByCost preserves the original order defined in the combo configuration, ensuring deterministic routing behavior.

How frequently does the pricing catalog update?

The providerRegistry refreshes periodically from upstream pricing feeds (such as LiteLLM), though exact intervals depend on your OmniRoute instance configuration. Each request uses the most recent catalog data available at execution time.

Can I combine cost-optimized routing with other strategies?

No, each combo uses a single strategy. However, you can create multiple combos with different strategies and implement client-side logic to choose between them based on additional criteria like latency or quality requirements.

Does cost-optimized routing account for output token costs?

The current implementation in targetSorters.ts primarily considers costPer1MTokens for input pricing. Output costs may be factored into the registry entries depending on your provider configuration, but the sorting logic focuses on the primary cost metric stored in the catalog.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}