How OmniRoute Cost Tracking Works with X-OmniRoute Headers and Quotas

OmniRoute computes per-request LLM costs using provider-specific pricing tables and exposes them via X-OmniRoute- HTTP response headers, then enforces monetary quotas by atomically incrementing SQLite counters against the X-OmniRoute-Response-Cost value.*

OmniRoute implements a transparent cost-tracking mechanism that calculates actual USD expenses for every LLM call. The gateway emits detailed telemetry through custom HTTP headers and leverages this data to enforce budget caps at the API-key level, ensuring that the value clients see matches the value used for quota accounting.

X-OmniRoute Response Headers Explained

Every successful (non-streaming) request returns a standardized set of X-OmniRoute-* headers containing cost telemetry. These headers are constructed by the buildOmniRouteResponseMetaHeaders function in src/domain/omnirouteResponseMeta.ts, which validates and encodes each value before injection into the HTTP response.

Core Cost Telemetry Headers

The following headers carry the primary billing data:

• X-OmniRoute-Response-Cost: The actual USD cost incurred, formatted as a fixed 10-decimal string (e.g., 0.0000045900). For cache hits, this value is forced to 0.0000000000.
• X-OmniRoute-Tokens-In: Input token count extracted from the upstream provider's prompt_tokens field.
• X-OmniRoute-Tokens-Out: Output token count extracted from completion_tokens or equivalent.
• X-OmniRoute-Provider: The provider alias (e.g., openai, anthropic) selected by the routing layer.
• X-OmniRoute-Model: The exact backend model name (e.g., gpt-4o-mini) resolved after combo-routing logic.

Operational Metadata Headers

Additional headers provide context for debugging and optimization:

• X-OmniRoute-Cache-Hit: Set to true when the semantic cache (open-sse/handlers/chatCore/semanticCache.ts) satisfies the request; omitted otherwise.
• X-OmniRoute-Cost-Saved: Present only on cache hits, indicating the USD value avoided by not hitting the upstream provider.
• X-OmniRoute-Latency-Ms: Total request processing time measured by the request-handler timing logic.
• X-OmniRoute-Fallback-Attempts: Count of routing retries due to circuit-breakers or connection cooldowns.

Cost Calculation Workflow

OmniRoute calculates costs through a three-stage pipeline that maps raw token usage to monetary values using provider-specific pricing catalogs.

Provider Pricing Configuration

Per-token costs are defined in src/shared/constants/providers.ts. For example, OpenAI's gpt-4o-mini pricing is configured as $0.000015 per 1K prompt tokens and $0.000060 per 1K completion tokens.

Token Extraction and Formula

After the upstream call completes, OmniRoute extracts usage statistics from the provider's JSON payload:

1. Input tokens: Read from prompt_tokens (or provider equivalent).
2. Output tokens: Read from completion_tokens (or provider equivalent).

The gateway applies the formula:

cost = (tokens_in × prompt_price_per_token) + (tokens_out × completion_price_per_token)

The result is rounded to 10 decimal places and stringified for the X-OmniRoute-Response-Cost header. For example, a request with 342 input tokens and 98 output tokens yields 0.0000045900.

Semantic Cache Handling

When the semantic cache satisfies a request, no upstream call occurs. The semanticCache.ts handler forces X-OmniRoute-Response-Cost to 0.0000000000 and populates X-OmniRoute-Cost-Saved with the computed value that would have been charged.

Quota Enforcement Architecture

OmniRoute uses the calculated cost to enforce budget limits at the API-key or OAuth account level through an integrated policy layer.

Policy Layer Integration

Quota rules reside in src/domain/policy/ and interact with SQLite storage managed by src/lib/db/quota.ts. After computing the cost, the gateway performs two operations atomically:

1. Increments the caller's consumed-cost counter by the X-OmniRoute-Response-Cost value.
2. Validates the total against the allocated budget.

Budget Exceedance Handling

If the accumulated cost exceeds the user's budget limit, OmniRoute returns an HTTP 429 error before forwarding the request upstream. This ensures that quota violations never trigger unnecessary provider charges.

Because quotas can be expressed as monetary caps (e.g., $10 per day) or token caps, the system internally translates token allocations to USD equivalents using the same provider pricing tables referenced during cost calculation.

Implementation Details

Building Response Headers

The core routine buildOmniRouteResponseMetaHeaders in src/domain/omnirouteResponseMeta.ts receives a metadata object and returns encoded header values. It handles edge cases such as percent-encoding non-ASCII model names and stripping control characters to ensure HTTP compliance.

Header names are defined as constants in src/shared/constants/headers.ts, ensuring consistency across the codebase.

Practical Integration Examples

Generating cost headers:

import { buildOmniRouteResponseMetaHeaders } from "@/domain/omnirouteResponseMeta";

const meta = {
  provider: "openai",
  model: "gpt-4o-mini",
  latencyMs: 125,
  tokensIn: 342,
  tokensOut: 98,
  responseCost: "0.0000045900",
  cacheHit: false,
};
const headers = buildOmniRouteResponseMetaHeaders(meta);
// Produces: { "X-OmniRoute-Provider": "openai", "X-OmniRoute-Model": "gpt-4o-mini", ... }

Enforcing quota limits:

const cost = Number(headers["X-OmniRoute-Response-Cost"]);
await quotaDB.incrementConsumedCost(apiKeyId, cost);
if (await quotaDB.isOverBudget(apiKeyId)) {
  return new Response("Quota exceeded", { status: 429 });
}

Summary

• OmniRoute injects X-OmniRoute-* headers into every response to provide transparent cost telemetry derived from actual upstream usage.
• X-OmniRoute-Response-Cost is calculated using provider-specific pricing from src/shared/constants/providers.ts and token counts from upstream usage payloads.
• Semantic cache hits force the cost to 0.0000000000 and expose savings via X-OmniRoute-Cost-Saved.
• Quota enforcement occurs in src/lib/db/quota.ts by incrementing SQLite counters against the response cost and returning HTTP 429 when budgets are exceeded.
• The buildOmniRouteResponseMetaHeaders function in src/domain/omnirouteResponseMeta.ts validates and encodes all header values to ensure HTTP compliance.

Frequently Asked Questions

What format does the X-OmniRoute-Response-Cost header use?

The header contains a USD value formatted as a fixed 10-decimal string (e.g., 0.0000045900). This precision ensures accurate aggregation for quota accounting across high-volume workloads while maintaining readability for client-side logging.

How does OmniRoute handle pricing differences between LLM providers?

Provider-specific per-token rates are hardcoded in src/shared/constants/providers.ts. The cost calculation engine multiplies extracted token counts by these rates, allowing mixed-provider workloads to consume quota from a unified budget pool denominated in USD.

What happens to cost headers when a response is served from cache?

When the semantic cache (open-sse/handlers/chatCore/semanticCache.ts) handles a request, X-OmniRoute-Response-Cost is set to 0.0000000000 and X-OmniRoute-Cost-Saved contains the avoided expense. This allows clients to track actual savings while ensuring quota consumption remains accurate.

How are quota limits configured and enforced in OmniRoute?

Quota rules are defined per-API-key or per-OAuth account in the policy layer (src/domain/policy/). Budgets can be specified as monetary caps or token allowances, with the system automatically converting token limits to USD equivalents using the provider pricing tables before enforcement in src/lib/db/quota.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}