How the OmniRoute Cost Calculation Engine Tracks Spending Across AI Providers

TLDR: OmniRoute tracks spending through a three-layer pipeline that pre‑estimates token costs, extracts exact provider‑billed amounts when available, and persists every transaction to SQLite for real‑time budget enforcement and dashboard reporting.

The OmniRoute cost calculation engine gives teams a provider‑agnostic way to monitor AI spend in real time. By combining predictive token estimation, exact cost extraction from provider APIs, and persistent budget windows, the system records and enforces spending limits across every routed request. As implemented in diegosouzapw/OmniRoute release v3.8.50, this architecture relies on pure functions, atomic batch writes, and SQLite aggregation to keep numbers accurate.

Pre‑Flight Token and Price Estimation

Before a request leaves the router, OmniRoute predicts its cost so the UI can display a preview. The src/shared/utils/costEstimator.ts module exports estimateTokens, estimateInputTokens, estimateCost, and preflightEstimate. These functions read per‑model pricing from the dashboard database or fall back to a hard‑coded pricing map. The estimator is side‑effect free: it counts tokens, multiplies by the provider’s rate, and returns a formatted string plus raw USD value without touching network or disk.

// Quick UI preview – estimate cost before routing
import { preflightEstimate } from '@/shared/utils/costEstimator';

const body = { messages: [{ role: 'user', content: 'Explain quantum entropy' }], max_tokens: 500 };
const model = 'gpt-4o';
const { formatted, totalCost } = preflightEstimate(body, model);
console.log(`≈ $${formatted} (${totalCost} USD)`);

Exact Provider Cost Extraction and Fallback Calculation

Once the provider responds, OmniRoute prioritizes real billed amounts over estimates. The src/lib/usage/costCalculator.ts file contains extractExactCostUsd and the async calculateCost function. If a provider returns a definitive value—such as xAI’s cost_in_usd_ticks field—that number is trusted immediately. When no exact cost exists, calculateCost loads pricing via @/lib/localDb#getPricingForModel and falls back to computeCostFromPricing, applying special rules like flat‑rate subscriptions, the Codex “fast” multiplier, and distinct cache‑read and cache‑creation rates.

// After a provider response – get the authoritative cost
import { calculateCost } from '@/lib/usage/costCalculator';

async function handleResponse(provider: string, model: string, usage: Record<string, number>) {
  const usd = await calculateCost(provider, model, usage, {
    provider,
    model,
    flatRateAsZero: true,          // hide per‑token estimate for subscription providers
  });
  console.log(`Provider‑billed cost: $${usd.toFixed(6)}`);
}

Budget Enforcement and Persistence

Every computed cost is written to an in‑memory batch writer and then flushed to SQLite for aggregation. src/domain/costRules.ts defines the getBudgetWindow helper, reset logic, and the cached budgets map that tracks running totals. The src/lib/spend/batchWriter.ts module—referenced inside costRules.ts—queues cost entries and periodically persists them to the cost_entries table in src/lib/db/domainState.ts. That same domain state module handles CRUD for budgets, cost entries, and reset logs.

// Query the current budget window for an API key
import { getBudgetWindow } from '@/domain/costRules';

const window = getBudgetWindow('daily', '03:00');
console.log(`Today started at ${new Date(window.periodStartAt).toISOString()}`);

The End‑to‑End Spend Tracking Flow

OmniRoute processes spend across five discrete stages that move from estimation to enforcement.

Step 1: Estimating Before Routing

When a request arrives, the chat handler invokes preflightEstimate to compute a token count and price preview. This value is surfaced to the UI so users see an approximate cost before any provider is billed.

Step 2: Calculating the Authoritative Cost

After execution, calculateCost checks for an exact USD value from the provider response. If none exists, it performs local token math using provider‑specific pricing and multipliers.

Step 3: Recording and Persisting Spend

The final USD amount is handed to spendBatchWriter, which buffers the entry in memory. The batch writer flushes the queue atomically to the cost_entries table via the SQLite layer in domainState.ts.

Step 4: Evaluating Budget Limits

On each request, costRules.ts loads the relevant budget from its cached map and updates daily, weekly, or monthly aggregates. If a defined limit is exceeded, the system rejects the request or emits a warning before routing continues.

Step 5: Surfacing Data to the Dashboard

The dashboard reads the same SQLite tables that power budget enforcement. Aggregated per‑provider spend and remaining quota are exposed through the domainState queries, ensuring the UI stays synchronized with the router’s internal ledger.

Summary

• Pre‑flight estimation in src/shared/utils/costEstimator.ts generates side‑effect‑free cost previews using token counts and provider pricing.
• Exact cost extraction in src/lib/usage/costCalculator.ts trusts provider‑billed values first, then falls back to local token math with support for flat rates, cache pricing, and Codex multipliers.
• Batch persistence through src/lib/spend/batchWriter.ts and src/lib/db/domainState.ts writes every entry to SQLite for durable aggregation.
• Budget enforcement in src/domain/costRules.ts evaluates daily, weekly, and monthly windows and rejects traffic when limits are breached.
• Dashboard parity means the UI consumes the same tables as the routing engine, giving a single source of truth for AI spend.

Frequently Asked Questions

How does OmniRoute handle providers that bill by subscription instead of tokens?

When a provider uses a flat‑rate model, calculateCost accepts the flatRateAsZero: true option. This suppresses per‑token estimates and records the provider’s exact billed amount—if any—or treats the cost as zero for budgeting purposes, avoiding misleading token math.

What happens when a provider returns an exact cost in its response?

OmniRoute always prefers the provider’s own figure. In src/lib/usage/costCalculator.ts, extractExactCostUsd scans the response for fields like xAI’s cost_in_usd_ticks. If an exact value is found, calculateCost returns it immediately without querying local pricing or computing tokens.

Where is historical spend data stored?

All cost entries and budget state live in SQLite. The src/lib/db/domainState.ts module manages the cost_entries, budgets, and reset logs, while src/lib/spend/batchWriter.ts buffers writes in memory and flushes them atomically to prevent data loss.

How are budget windows calculated?

src/domain/costRules.ts exports getBudgetWindow, which accepts a cycle type—daily, weekly, or monthly—and a reset offset such as '03:00'. It returns a periodStartAt timestamp that aligns spend aggregation to the configured window, letting quotas reset on predictable schedules.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}