OmniRoute Live Telemetry: Complete Real-Time Observability Guide

OmniRoute exposes live telemetry through the /api/monitoring/health endpoint, returning a comprehensive JSON payload that aggregates diagnostics from every core subsystem with a 1-second refresh rate.

OmniRoute live telemetry provides operators with near-real-time visibility into router health, provider performance, and resilience mechanisms. The telemetry system aggregates data across circuit breakers, connection pools, rate limits, and adaptive admission controls through a single HTTP endpoint. All metrics are cached with a 1000ms TTL for rapid polling without performance impact.

Accessing the Live Telemetry Endpoint

The health endpoint serves as the unified entry point for OmniRoute observability data.

GET Requests

# Full telemetry with authentication (requires admin token)

curl -H "Authorization: Bearer <admin-token>" \
     https://localhost:20128/api/monitoring/health

# Public health-check view (no authentication)

curl https://localhost:20128/api/monitoring/health

The authenticated view returns the complete telemetry payload. The unauthenticated view returns a restricted subset suitable for load balancer health checks.

DELETE Requests

# Reset all circuit breakers to CLOSED state (admin only)

curl -X DELETE -H "Authorization: Bearer <admin-token>" \
     https://localhost:20128/api/monitoring/health

This operation instantly clears failure states across all providers.

Core Telemetry Categories

OmniRoute live telemetry surfaces 17 distinct diagnostic categories. Each is populated by specific functions in src/lib/monitoring/observability.ts.

System and Process Metrics

The buildHealthPayload function aggregates fundamental runtime data:

• Version and build SHA (populated via src/lib/monitoring/buildSha.ts)
• Node.js runtime version
• Process uptime and PID
• Current memory usage
• Platform identifier

These fields appear at the root of the JSON response.

Circuit Breaker States

Per-provider resilience status reveals whether upstream providers are operational:

Field	Description
state	OPEN, CLOSED, DEGRADED, or HALF-OPEN
failureCount	Accumulated failures since last success
lastFailureAt	ISO timestamp of most recent failure
retryAfterMs	Milliseconds until next retry attempt

Populated by buildHealthPayload in lines 23-44 of observability.ts, with data sourced from the circuit breaker registry and providerHealth tracking.

Connection Health

The summarizeConnectionCooldown function (lines 8-41) reports connection pool status per provider:

• Total active connections
• Connections in cooldown state
• Time until earliest retry

This helps diagnose provider congestion and backpressure conditions.

Codex Account Pools

For OpenAI Codex integrations, summarizeCodexAccountPools (lines 52-87) exposes:

• Available accounts in pool
• Limited/quota-bound accounts
• Quota observations per account
• Soonest retry timestamp

Rate Limiting and Adaptive Controls

OmniRoute live telemetry includes multiple interrelated limit systems:

• rateLimitStatus (lines 100-101) — Current global counters per provider
• learnedLimits (lines 104-105) — Adaptive thresholds derived from traffic patterns
• lockouts (lines 102-103) — Per-model lockout counts for quota and permission errors

Session and Admission Telemetry

Active request handling surfaces through:

• buildSessionsSummary (lines 30-46) — Active session count, sticky-bound connections, top-8 sessions with request volume and age metrics
• projectAdaptiveAdmissionSummary (lines 92-118) — Runtime admission state including mode, limits, utilization percentage, pressure score, and shutdown flag
• projectChatAdmissionSummary (lines 44-61) — Structural chat admission health: heavy-lease status, waiting queue depth, byte budget consumption, and shed counts

Additional Telemetry Fields

Category	Source	Purpose
Local Provider Health	localProviders (line 104)	Self-hosted LLM status
Quota Monitor	quotaMonitorSummary / quotaMonitorMonitors (lines 106-108)	Session exhaustion tracking with 8-snapshot history
Credential Health	credentialHealth (line 108)	API key validity totals (healthy, failed, unknown, stale)
Deduplication	dedup.inflightRequests (line 114)	Active request deduplication entry count
Cryptography	cryptography.status (lines 16-21)	Storage encryption key validity
Setup Completion	setupComplete (line 23)	Configuration wizard finish flag

Consuming Telemetry Programmatically

Node.js Client Example

import fetch from "node-fetch";

interface HealthPayload {
  version: string;
  buildSha: string;
  uptime: number;
  activeConnections: number;
  providerHealth: Record<string, {
    state: "OPEN" | "CLOSED" | "DEGRADED" | "HALF-OPEN";
    failureCount: number;
    lastFailureAt?: string;
  }>;
  rateLimitStatus: Record<string, number>;
  adaptiveAdmission: {
    mode: string;
    utilization: number;
    pressure: number;
    shutdown: boolean;
  };
}

async function fetchHealth(): Promise<HealthPayload> {
  const res = await fetch("http://localhost:20128/api/monitoring/health", {
    headers: process.env.ADMIN_TOKEN 
      ? { Authorization: `Bearer ${process.env.ADMIN_TOKEN}` }
      : undefined
  });
  
  if (!res.ok) throw new Error(`Health check failed: ${res.status}`);
  return res.json();
}

// Alert on degraded providers
fetchHealth().then(payload => {
  const degraded = Object.entries(payload.providerHealth)
    .filter(([, h]) => h.state === "DEGRADED" || h.state === "OPEN");
  
  if (degraded.length > 0) {
    console.warn("Provider issues detected:", degraded.map(([name]) => name));
  }
});

Source Code Reference

OmniRoute live telemetry implementation spans these key files:

File	Responsibility
src/app/api/monitoring/health/route.ts	HTTP GET/DELETE handlers, response caching, authentication gating
src/lib/monitoring/observability.ts	Core aggregation functions: buildHealthPayload, summarizeConnectionCooldown, summarizeCodexAccountPools, buildSessionsSummary, projectAdaptiveAdmissionSummary, projectChatAdmissionSummary
src/lib/monitoring/buildSha.ts	Git SHA extraction for deployment tracking
src/lib/monitoring/providerHealthMatrix.ts	Provider-level dashboard matrices
src/lib/monitoring/providerHealthAutopilot.ts	Autopilot health calculations
src/lib/monitoring/comboHealthAutopilot.ts	Combined health scoring for UI components

Summary

• Single endpoint: OmniRoute live telemetry consolidates 17 diagnostic categories at /api/monitoring/health
• Refresh rate: All metrics update every 1000ms with server-side caching
• Authentication tiers: Full payload requires admin token; reduced public view available unauthenticated
• Operational controls: DELETE requests reset all circuit breakers instantly
• Key source: src/lib/monitoring/observability.ts contains the primary aggregation logic with function-specific line ranges preserved in the codebase

Frequently Asked Questions

What is the refresh rate for OmniRoute live telemetry?

The telemetry payload regenerates every 1000 milliseconds (1 second) with a TTL-enforced cache. This design balances real-time observability against measurement overhead. The caching layer in src/app/api/monitoring/health/route.ts prevents redundant calculation during high-frequency polling.

How do I reset a stuck provider without restarting OmniRoute?

Send an authenticated DELETE request to /api/monitoring/health. This resets all circuit breakers to the CLOSED state, immediately clearing OPEN or DEGRADED conditions across every provider. Individual provider reset is not exposed; the operation is global by design.

Why does the unauthenticated health endpoint return fewer fields?

The public view omits sensitive operational data including credential health, detailed quota monitors, and cryptographic status. This allows safe exposure to load balancers and external health checkers without leaking deployment internals. Authentication scope validation occurs in src/app/api/monitoring/health/route.ts.

Can I export OmniRoute live telemetry to Prometheus or Datadog?

Not natively. The endpoint returns JSON over HTTP; integration requires a translation layer. The structured payload in buildHealthPayload uses consistent naming conventions suitable for metric extraction. Many operators deploy a sidecar that polls the endpoint and emits to their observability platform.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}