How OmniRoute Implements Cost Telemetry Using X-OmniRoute Headers

OmniRoute tracks usage-based cost information for every request and propagates it to callers via custom HTTP headers prefixed with X-OmniRoute-, including token counts, latency, and normalized USD costs.

The diegosouzapw/OmniRoute repository provides an AI gateway that normalizes requests across multiple LLM providers. To ensure transparent billing and observability, the gateway implements cost telemetry using a standardized set of X-OmniRoute-* response headers that expose token usage, monetary cost, and performance metrics for every successful request.

The Telemetry Implementation Pipeline

Extracting Token Counts from Provider Responses

After an upstream provider returns a response, OmniRoute extracts token counts from the provider-specific usage payload. The helper getOmniRouteTokenCounts in src/domain/omnirouteResponseMeta.ts normalizes the various field names used by different providers (such as prompt_tokens and completion_tokens) into a consistent internal format.

Normalizing Cost to Fixed Precision

The raw cost in USD is normalized to a finite number and rendered with fixed 10-decimal precision by the formatOmniRouteCost function. This ensures that monetary values are consistently represented across all provider responses, preventing floating-point discrepancies in billing calculations.

Sanitizing Header Values for HTTP Compliance

Header strings must be ASCII-safe to comply with HTTP standards. The toHeaderValue utility removes control characters, validates pure ASCII content, and URL-encodes Unicode strings when necessary. This sanitization step guarantees that all telemetry data can be safely transmitted as HTTP header values.

Assembling the Complete Telemetry Payload

The buildOmniRouteResponseMetaHeaders function constructs a plain object containing all telemetry headers defined in src/shared/constants/headers.ts. This payload includes:

• Cache status: X-OmniRoute-Cache and X-OmniRoute-Cache-Hit
• Performance: X-OmniRoute-Latency-Ms
• Cost: X-OmniRoute-Response-Cost (the formatted USD value)
• Usage: X-OmniRoute-Tokens-In and X-OmniRoute-Tokens-Out
• Routing: X-OmniRoute-Model and X-OmniRoute-Provider
• Optional metadata: X-OmniRoute-Request-Id, X-OmniRoute-Cost-Saved (for cache-hit savings), and X-OmniRoute-Fallback-Attempts (retry count)

Attaching Headers to the HTTP Response

For non-streaming responses, the attachOmniRouteMetaHeaders helper mutates an existing Headers object or plain record with the telemetry map. For opaque Response objects (such as audio streams), attachOmniRouteMetaToResponse safely adds headers by cloning the response when direct mutation fails. All successful OmniRoute routes invoke these helpers to ensure every response carries the complete telemetry set.

Practical Implementation Examples

Building Telemetry Headers in a Handler

import { buildOmniRouteResponseMetaHeaders } from "@/domain/omnirouteResponseMeta";

const metaHeaders = buildOmniRouteResponseMetaHeaders({
  cacheHit: false,
  costUsd: upstreamResult.costUsd,
  latencyMs: Date.now() - startTime,
  model: upstreamResult.model,
  provider: upstreamResult.provider,
  usage: upstreamResult.usage,
});

// Returns Record<string, string> with all X-OmniRoute-* entries

Attaching Telemetry to Response Objects

import { attachOmniRouteMetaHeaders, attachOmniRouteMetaToResponse } from "@/domain/omnirouteResponseMeta";

// For standard Headers object
const responseHeaders = new Headers();
attachOmniRouteMetaHeaders(responseHeaders, metaData);

// For streaming or opaque Response objects
const enrichedResponse = attachOmniRouteMetaToResponse(rawResponse, metaData);

Reading Cost Telemetry on the Client

fetch("/api/v1/chat/completions", { method: "POST", body: JSON.stringify(payload) })
  .then(res => {
    console.log("Cost:", res.headers.get("X-OmniRoute-Response-Cost"));
    console.log("Tokens In:", res.headers.get("X-OmniRoute-Tokens-In"));
    console.log("Tokens Out:", res.headers.get("X-OmniRoute-Tokens-Out"));
    console.log("Provider:", res.headers.get("X-OmniRoute-Provider"));
  });

Summary

• OmniRoute extracts provider-specific token usage via getOmniRouteTokenCounts in src/domain/omnirouteResponseMeta.ts.
• Costs are normalized to 10-decimal precision using formatOmniRouteCost to ensure billing consistency.
• Header values are sanitized by toHeaderValue to guarantee ASCII-safe HTTP compliance.
• The buildOmniRouteResponseMetaHeaders function assembles complete telemetry payloads with canonical header names from src/shared/constants/headers.ts.
• Header attachment helpers support both standard Headers objects and opaque Response streams via attachOmniRouteMetaHeaders and attachOmniRouteMetaToResponse.

Frequently Asked Questions

What specific cost information does OmniRoute expose in headers?

OmniRoute exposes the total request cost in USD via the X-OmniRoute-Response-Cost header, formatted to 10 decimal places. When a cache hit occurs, the X-OmniRoute-Cost-Saved header indicates the monetary amount saved by avoiding an upstream provider call.

How does OmniRoute handle different provider token count field names?

The getOmniRouteTokenCounts helper in src/domain/omnirouteResponseMeta.ts normalizes provider-specific field names (such as prompt_tokens or input_tokens) into standardized tokensIn and tokensOut values, ensuring consistent X-OmniRoute-Tokens-In and X-OmniRoute-Tokens-Out headers regardless of the upstream provider.

Can Unicode characters in model names break the telemetry headers?

No. The toHeaderValue utility sanitizes all header values by removing control characters and URL-encoding any Unicode content, ensuring that X-OmniRoute-Model and other headers remain valid ASCII strings compliant with HTTP specifications.

How is cost telemetry attached to streaming responses?

For streaming or binary responses where the Response object might be opaque, OmniRoute uses attachOmniRouteMetaToResponse to safely inject headers. If direct mutation fails, the function clones the response with the new headers attached, ensuring telemetry is preserved without disrupting the stream.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}