How OmniRoute's CostEstimator Tracks Usage and Calculates Provider Quotas

OmniRoute separates deterministic cost estimation from asynchronous quota tracking, using pure functions to calculate USD costs from token counts while maintaining cached provider quota windows to prevent routing requests to exhausted accounts.

OmniRoute's costEstimator is a robust subsystem within the diegosouzapw/OmniRoute repository that handles the dual challenges of predicting request expenses and enforcing provider limits. It distinguishes between cost estimation—a synchronous calculation based on token pricing—and quota tracking—an asynchronous service that monitors real-time usage limits across multiple AI providers.

Token-Level Cost Calculation

The foundation of OmniRoute's budgeting lies in src/lib/usage/costCalculator.ts, which exports the core calculateCost function. This module treats price calculation as a pure, deterministic operation that requires only token counts and a pricing record.

When processing usage, the calculator first attempts to extract an exact provider-reported cost via extractExactCostUsd. For example, xAI includes a cost_in_usd_ticks field in its responses; when present, this value overrides token-based estimates entirely:

• Exact cost extraction: Checks for cost_in_usd_ticks before falling back to calculation
• Pricing lookup: Uses getPricingForModel to retrieve per-million-token rates for input, output, cached, reasoning, and cache-creation tokens
• Flat-rate handling: Supports the flatRateAsZero option for subscription-based providers

The computeCostFromPricing function multiplies token counts by these rates, ensuring accurate billing even for complex pricing tiers involving cached context or reasoning tokens.

Modality-Specific Cost Helpers

Beyond text tokens, OmniRoute handles multimodal requests through specialized helpers in the same calculator module. The calculateModalCost function dispatches to provider-specific implementations based on content type:

• computeImageCost: Calculates expenses for image generation using output_cost_per_image fields
• computeAudioCost: Handles transcription and text-to-speech pricing
• computeVideoCost: Processes video generation or analysis rates
• computeRerankCost: Manages rerank unit pricing for semantic search operations

Each helper reads from the normalized pricing record loaded via getPricingForModel, ensuring consistent cost attribution across diverse AI modalities.

Model Name Normalization

Accurate pricing requires matching provider model identifiers to internal database records. The normalizeModelName function in src/lib/usage/costCalculator.ts strips path prefixes and provider-specific suffixes (such as Codex effort indicators via stripCodexEffortSuffix) before querying the local pricing database. This normalization guarantees that variants like openai/gpt-4o and gpt-4o resolve to the correct pricing tier.

Quota Fetching and Caching

Quota tracking operates asynchronously through the pre-flight system defined in open-sse/services/quotaPreflight.ts. This service periodically queries providers for current usage windows (daily, weekly, monthly) and parses responses into a standardized QuotaCacheView containing quotaPercent and resetAt timestamps.

The caching architecture employs two layers:

1. In-memory cache: open-sse/services/quotaMonitor.ts maintains hot quota data for rapid routing decisions
2. Persistent storage: src/lib/db/quotaSnapshots.ts persists quota states to the SQLite quota_snapshots table, ensuring durability across service restarts

When providers return quota errors (HTTP 429 or 403), the quotaPreflightUnavailableUntil utility parses Retry-After or resetAt headers into human-readable availability windows, temporarily removing exhausted connections from the routing pool.

Quota-Aware Routing

The combo router integrates quota data into its scoring algorithm via open-sse/services/combo/quotaScoring.ts. Functions isQuotaExhaustedForRequest and getConnectionQuotaHeadroomPercent evaluate cached quota snapshots to calculate a quota-headroom penalty. This penalty reduces the selection probability for connections approaching their limits, effectively load-balancing traffic across accounts based on remaining capacity rather than just latency or cost.

This integration ensures that OmniRoute proactively avoids providers nearing quota exhaustion, preventing request failures before they occur.

Analytics and Reporting

OmniRoute aggregates historical usage data in the daily_usage_summary table, tracking per-provider and per-model totals for requests, input tokens, output tokens, and calculated costs. The API routes under src/app/api/usage/analytics/ expose this data for dashboard visualization, enabling real-time monitoring of spending trends and quota utilization percentages across all configured accounts.

Practical Implementation

The following examples demonstrate common costEstimator operations.

Calculate the cost of a chat request using token counts:

import { calculateCost } from "@/lib/usage/costCalculator";

const provider = "openai";
const model = "gpt-4o";
const usage = {
  input: 12_345,
  output: 4_567,
  // optional: cost_in_usd_ticks – when present (xAI) it wins over the token estimate
};

const usd = await calculateCost(provider, model, usage);
// → e.g. 0.0142 USD

Compute costs for multimodal content like image generation:

import { calculateModalCost } from "@/lib/usage/costCalculator";

const modality = "image";
const usage = { n: 3 };               // three images requested
const cost = await calculateModalCost(modality, "openai", "dall-e-3", usage);
// → per-image price × 3

Check quota status before routing a request:

import { preflightQuota } from "@omniroute/open-sse/services/quotaPreflight";

const connectionId = "conn-123";
const preflight = await preflightQuota(connectionId, { provider: "anthropic", model: "claude-3-5-sonnet" });

if (preflight.quotaPercent !== undefined) {
  console.log(`Quota used: ${(preflight.quotaPercent * 100).toFixed(1)}%`);
  console.log(`Reset at: ${preflight.resetAt}`);
}

Retrieve aggregated daily costs for dashboard reporting:

import { getDailyUsageSummary } from "@/lib/db/usageAnalytics";

const summary = await getDailyUsageSummary();   // reads `daily_usage_summary` table
summary.byProvider.forEach(p => {
  console.log(`${p.provider}: $${p.cost.toFixed(2)} today`);
});

Summary

• Cost calculation is deterministic and synchronous, residing in src/lib/usage/costCalculator.ts with support for both token-based and exact-cost extraction
• Quota tracking is asynchronous and cached, implemented via open-sse/services/quotaPreflight.ts and quotaMonitor.ts with SQLite persistence in src/lib/db/quotaSnapshots.ts
• Model normalization functions normalizeModelName and stripCodexEffortSuffix ensure pricing lookups succeed despite provider naming inconsistencies
• Routing integration uses quota headroom scoring in open-sse/services/combo/quotaScoring.ts to avoid exhausted accounts before requests are dispatched
• Analytics aggregation populates the daily_usage_summary table for trend analysis and budgeting dashboards accessed via src/app/api/usage/analytics/

Frequently Asked Questions

How does OmniRoute handle providers that report exact costs rather than token counts?

When providers like xAI return a cost_in_usd_ticks field in their response, the extractExactCostUsd function in src/lib/usage/costCalculator.ts captures this value and bypasses token-based calculations entirely. This ensures billing accuracy when providers use proprietary pricing models that don't align with simple per-token rates.

What happens when a provider returns a quota error mid-request?

If a provider returns a 429 or 403 error indicating quota exhaustion, OmniRoute's quotaPreflightUnavailableUntil function parses the Retry-After or resetAt headers to determine when the account will be available again. This timestamp is stored in the quota cache, and the connection receives a routing penalty via getConnectionQuotaHeadroomPercent in open-sse/services/combo/quotaScoring.ts, diverting traffic to alternative providers until the quota resets.

Can cost estimation run entirely offline without provider API calls?

Yes, the calculateCost function is a pure operation that requires only token counts and a local pricing record from the database. Since it uses getPricingForModel to read cached pricing data rather than querying live APIs, cost estimation can compute projected expenses offline, making it suitable for budget forecasting and request pre-validation before any network traffic is sent to providers.

How does the quota caching strategy balance data freshness with performance?

The system employs a two-tier cache where open-sse/services/quotaMonitor.ts maintains an in-memory QuotaCacheView for millisecond-level routing decisions, while src/lib/db/quotaSnapshots.ts persists data to SQLite for durability. The pre-flight service in open-sse/services/quotaPreflight.ts periodically refreshes these caches by querying provider APIs, ensuring routing decisions use recent data without introducing latency on the critical path of request processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}