How Usage Is Tracked Within OmniRoute: A Deep Dive into Request Analytics

OmniRoute tracks every AI request through a three-layer architecture combining in-memory pending counters, persistent SQLite storage, and aggregated analytics to deliver real-time cost calculations and dashboard statistics across all providers.

OmniRoute is an open-source request router for large language models that requires precise visibility into token consumption and operational costs. Understanding how usage is tracked within OmniRoute reveals a sophisticated pipeline that captures every request from initiation through completion, storing granular metrics while maintaining low-latency in-memory state for active requests.

The Three-Layer Tracking Architecture

The system operates through tightly coupled layers that transition data from ephemeral runtime state to persistent storage and finally to aggregated analytics:

• In-Memory Pending Tracking – Maintains real-time counters and metadata while requests are in flight using src/lib/usage/usageHistory.ts
• Persistent Storage – Writes normalized records to the SQLite usage_history table upon completion via saveRequestUsage()
• Aggregated Analytics – Computes dashboard statistics, cost breakdowns, and time-series data through src/lib/usage/usageStats.ts

Real-Time Pending Request Tracking

When a request enters the router, OmniRoute immediately begins tracking it in memory to provide instantaneous visibility into active load.

Starting a Pending Request

The trackPendingRequest() function in src/lib/usage/usageHistory.ts initializes tracking by constructing a safe model key (e.g., "gpt-4 (openai)"), incrementing counters in pendingRequests.byModel and pendingRequests.byAccount[connectionId], and generating a cryptographically secure pending ID. This ID is stored in pendingById for O(1) lookup during the request lifecycle.

import { trackPendingRequest } from '@/lib/usage/usageHistory';

const pendingId = trackPendingRequest(
  "claude-2.1",          // model
  "anthropic",           // provider
  connectionId,          // provider connection id
  true,                  // started?
  { clientEndpoint: "/v1/chat/completions" }
);

Background Cleanup and Memory Limits

To prevent memory leaks, ensurePendingSweepTimer() runs a background sweep that removes entries older than MAX_PENDING_REQUEST_AGE_MS (default 1 hour) and caps total stored details at MAX_PENDING_DETAILS. These safeguards ensure that orphaned pending records from crashed requests do not accumulate indefinitely.

Persistent Storage with Deduplication

Once a request completes—successfully or with an error—the router finalizes tracking by persisting a detailed record to SQLite.

The saveRequestUsage workflow

The saveRequestUsage() function in src/lib/usage/usageHistory.ts performs several critical operations:

1. Normalizes timestamps and service-tier metadata
2. Extracts token counts using helpers from src/lib/usage/tokenAccounting.ts (getLoggedInputTokens, getLoggedOutputTokens)
3. Resolves account identity via resolveUsageAccountIdentity()
4. Executes deduplication logic that skips inserts if a row with identical timestamp, provider, model, connection, API key, and token counts already exists
5. Inserts the record into usage_history with latency, error codes, and endpoint details
6. Emits the usageRecorded event for quota enforcement subsystems

import { saveRequestUsage } from '@/lib/usage/usageHistory';

await saveRequestUsage({
  provider: "openai",
  model: "gpt-4",
  connectionId: connId,
  tokens: { input: 150, output: 450 },
  success: true,
  latencyMs: 210,
  errorCode: null,
  endpoint: "/v1/chat/completions"
});

The database schema is defined in src/lib/db/migrations/ and supports high-cardinality fields like api_key_id, connection_id, and token-specific columns for input, output, and cache reads.

Aggregated Analytics and Cost Calculation

The dashboard and API endpoints consume processed statistics generated by src/lib/usage/usageStats.ts.

Building Unified Statistics Queries

The getUsageStats() function determines whether aggregation is enabled and constructs a UNION query via buildUsageSourceSql(). This query combines recent raw rows from usage_history with pre-aggregated historical data from daily_usage_summary to balance query performance with data granularity.

Cost Computation and Time Bucketing

The system calculates costs through calculateAggregateCost() in src/lib/usage/costCalculator.ts, which applies provider-specific pricing tables to token counts. The resulting statistics object includes:

• Total metrics: totalRequests, totalPromptTokens, totalCompletionTokens, totalCost
• Dimensional breakdowns: byProvider, byModel, byAccount, byApiKey
• Time-series data: 10-minute buckets showing request velocity and token throughput
• Active state: Merged pending request counters from the in-memory layer

import { getUsageStats } from '@/lib/usage/usageStats';

const stats = await getUsageStats();
console.log(`Total USD spent: $${stats.totalCost.toFixed(2)}`);
console.log(`Requests in last 10 min: ${stats.last10Minutes.reduce((s,b) => s+b.requests, 0)}`);

Supporting Infrastructure

Several specialized modules ensure accurate tracking across diverse provider implementations:

• Token Accounting – src/lib/usage/tokenAccounting.ts normalizes provider-specific token fields (OpenAI's prompt_tokens, Anthropic's input_tokens, caching metrics) into a unified schema for storage and cost calculation.
• Cost Calculator – src/lib/usage/costCalculator.ts maintains per-model pricing tables and conversion logic to generate accurate USD estimates.
• Account Identity – src/lib/usage/accountIdentity.ts resolves connection IDs into human-readable account names for the byAccount analytics breakdown.
• Event System – src/lib/usage/usageEvents.ts emits usageRecorded events that trigger downstream quota enforcement and rate-limiting logic.

Summary

• OmniRoute employs a three-layer architecture combining in-memory pending tracking, SQLite persistence, and aggregated analytics.
• In-memory tracking uses trackPendingRequest() with automatic cleanup timers to monitor active requests without database overhead.
• Deduplication logic in saveRequestUsage() prevents duplicate entries based on timestamp, provider, model, and token counts.
• Cost calculation relies on provider-specific pricing tables in costCalculator.ts to convert token counts to USD.
• The analytics pipeline unions raw and pre-aggregated data to serve dashboard statistics and 10-minute velocity buckets.

Frequently Asked Questions

How does OmniRoute prevent duplicate usage entries?

OmniRoute implements deduplication within saveRequestUsage() in src/lib/usage/usageHistory.ts by checking for existing rows matching the same timestamp, provider, model, connection, API key, and token counts before inserting. If a match exists, the system either skips the insert or backfills missing fields like endpoint rather than creating redundant records.

What database does OmniRoute use for usage tracking?

OmniRoute uses SQLite for all usage persistence, storing data in the usage_history table with a schema defined in src/lib/db/migrations/. For analytics performance, the system also maintains a daily_usage_summary table containing pre-aggregated historical data that the dashboard queries via UNION operations with recent raw records.

How are costs calculated for different AI providers?

Costs are calculated in src/lib/usage/costCalculator.ts using provider-specific pricing tables mapped to model names. The calculateCost() function multiples input and output token counts by the respective per-million-token rates for each provider, handling tiered pricing and cached token discounts where applicable.

Can I query usage statistics programmatically?

Yes, the getUsageStats() function exported from src/lib/usage/usageStats.ts provides programmatic access to aggregated statistics including total costs, token counts, dimensional breakdowns by model and provider, and 10-minute time buckets. This powers the internal dashboard and can be consumed by external monitoring systems through the /api/v1/usage/analytics endpoint.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}