How OmniRoute Minimizes Per‑Request Spending with Cost‑Optimized Routing

OmniRoute minimizes per‑request spending by selecting the cheapest viable connection through a priority‑based sorting algorithm that orders candidates by estimated cost.

OmniRoute's cost‑optimized routing is a built‑in strategy that automatically routes each request to the least‑expensive available provider. This article explains how the system works under the hood, using actual implementation details from the diegosouzapw/OmniRoute repository.

How Cost‑Optimized Routing Works

When you enable cost‑optimized routing, OmniRoute executes a three‑step process that guarantees the cheapest valid connection handles your request.

Priority‑Based Connection Selection

The core logic lives in src/sse/services/auth.ts. The getProviderCredentialsWithQuotaPreflight function checks the strategy argument, and when it equals "cost-optimized", the code sorts eligible connections by their priority field and selects the first entry.

// From src/sse/services/auth.ts (lines 1623-1629)
// When strategy === "cost-optimized":
const sortedConnections = eligibleConnections.sort(
  (a, b) => a.priority - b.priority
);
const selectedConnection = sortedConnections[0];

Lower priority values indicate cheaper or preferred connections. The sorting happens after filtering for health status and quota availability, so cost optimization never violates reliability constraints.

Where Priority Values Originate

Connection priorities come from two sources in src/lib/db/connections.ts:

• Explicit priority field – Set manually or via API based on known provider pricing tiers
• Computed priority from cost estimation – Generated dynamically using the Cost Estimator utility

The database schema stores priority as a numeric field on every connection record, making sorting operations fast and deterministic.

Pre‑Flight Cost Estimation

src/shared/utils/costEstimator.ts provides a pure function that calculates expected USD cost from token counts and model pricing data:

// Example: estimate cost before assigning priority
import { estimateCost } from "@/shared/utils/costEstimator";

async function assignPriority(
  connectionId: string,
  model: string,
  tokensIn: number,
  tokensOut: number
) {
  const costUsd = await estimateCost(model, tokensIn, tokensOut);
  // Convert cost to priority: lower cost → lower priority number
  const priority = Math.round(costUsd * 1_000); // $0.001 → 1
  await updateConnection(connectionId, { priority });
}

When a connection lacks an explicit priority, OmniRoute uses this estimator to derive one automatically. This ensures newer or dynamically priced providers participate in cost‑optimized routing without manual configuration.

Routing Strategy Registration

The "cost-optimized" label is formally declared in src/shared/constants/routingStrategies.ts:

// From routingStrategies.ts
export const ROUTING_STRATEGIES = [
  // ... other strategies
  {
    value: "cost-optimized",
    label: "Cost-Optimized",
    description: "Route to the cheapest available provider"
  }
] as const;

This registration exposes the strategy through both the REST API and the management UI, letting users select cost‑optimized routing without understanding the underlying priority mechanics.

Practical Implementation

Requesting a Cost‑Optimized Credential

Use the following pattern to fetch credentials with cost‑optimized routing enabled:

import { getProviderCredentialsWithQuotaPreflight } from "@/sse/services/auth";

async function fetchCheapCredential(provider: string, model: string) {
  const cred = await getProviderCredentialsWithQuotaPreflight(
    provider,
    null,                     // no connection to exclude
    null,                     // allow all connections
    model,
    { strategy: "cost-optimized" }  // enables cost‑optimized routing
  );
  return cred; // guaranteed lowest‑priority (cheapest) connection
}

The function returns the first connection after priority sorting, which represents the minimal per‑request spend available for your constraints.

Cost Calculation Pipeline

src/lib/usage/costCalculator.ts centralizes all cost arithmetic. Both the estimator (for priority assignment) and post‑request analytics consume this module, ensuring consistent pricing logic across the platform.

Key Architectural Benefits

• No manual quota juggling – The system automatically prefers cheaper providers without user intervention
• Respects hard constraints – Health checks and quota limits filter connections before cost sorting occurs
• Adaptive to pricing changes – Dynamic priority recalculation responds to provider price updates
• Transparent cost tracking – The same calculator used for routing feeds real‑time spend dashboards

Summary

• OmniRoute's cost‑optimized routing minimizes per‑request spending by sorting connections by priority in src/sse/services/auth.ts
• Lower priority values indicate cheaper options, with values sourced from either explicit configuration or the estimateCost utility in src/shared/utils/costEstimator.ts
• Pre‑flight cost estimation converts dollar amounts to comparable priority integers, enabling automatic participation of new providers
• The "cost-optimized" strategy is registered in src/shared/constants/routingStrategies.ts and accessible via API and UI
• Cost calculation logic is centralized in src/lib/usage/costCalculator.ts for consistency across routing and analytics

Frequently Asked Questions

What happens if no connection has a priority set?

OmniRoute falls back to the cost estimator. For any connection lacking an explicit priority, the system calls estimateCost() using the target model and typical token counts, then converts that dollar value to a priority number. This ensures cost‑optimized routing still functions with incomplete configuration.

Does cost‑optimized routing ignore provider reliability?

No. The sorting by priority only occurs after health checks and quota validation filter out unavailable connections. A cheap connection with failed health checks never reaches the priority sort stage, so reliability constraints always take precedence over cost.

How does OmniRoute handle real‑time pricing changes?

The priority field can be updated dynamically via updateConnection() in src/lib/db/connections.ts. Automated jobs or webhooks from providers can refresh priorities using fresh cost estimates, causing subsequent requests to respect new pricing without restarting the router.

Can I combine cost‑optimized routing with other strategies?

The current implementation treats "cost-optimized" as a discrete strategy value. You cannot blend it with latency‑or‑round‑robin approaches in a single request. However, you can implement tiered logic in your application code—falling back to cost‑optimized only when faster strategies exhaust their quotas.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}