# OmniRoute Live Telemetry: Complete Real-Time Observability Guide

> Discover OmniRoute live telemetry with a complete real-time observability guide. Access health diagnostics from every core subsystem every second via the /api/monitoring/health endpoint.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-09-01

---

**OmniRoute exposes live telemetry through the `/api/monitoring/health` endpoint, returning a comprehensive JSON payload that aggregates diagnostics from every core subsystem with a 1-second refresh rate.**

OmniRoute live telemetry provides operators with near-real-time visibility into router health, provider performance, and resilience mechanisms. The telemetry system aggregates data across circuit breakers, connection pools, rate limits, and adaptive admission controls through a single HTTP endpoint. All metrics are cached with a 1000ms TTL for rapid polling without performance impact.

## Accessing the Live Telemetry Endpoint

The health endpoint serves as the unified entry point for OmniRoute observability data.

### GET Requests

```bash

# Full telemetry with authentication (requires admin token)

curl -H "Authorization: Bearer <admin-token>" \
     https://localhost:20128/api/monitoring/health

# Public health-check view (no authentication)

curl https://localhost:20128/api/monitoring/health

```

The authenticated view returns the complete telemetry payload. The unauthenticated view returns a restricted subset suitable for load balancer health checks.

### DELETE Requests

```bash

# Reset all circuit breakers to CLOSED state (admin only)

curl -X DELETE -H "Authorization: Bearer <admin-token>" \
     https://localhost:20128/api/monitoring/health

```

This operation instantly clears failure states across all providers.

## Core Telemetry Categories

OmniRoute live telemetry surfaces 17 distinct diagnostic categories. Each is populated by specific functions in [`src/lib/monitoring/observability.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/observability.ts).

### System and Process Metrics

The **`buildHealthPayload`** function aggregates fundamental runtime data:

- Version and build SHA (populated via [`src/lib/monitoring/buildSha.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/buildSha.ts))
- Node.js runtime version
- Process uptime and PID
- Current memory usage
- Platform identifier

These fields appear at the root of the JSON response.

### Circuit Breaker States

Per-provider resilience status reveals whether upstream providers are operational:

| Field | Description |
|-------|-------------|
| `state` | OPEN, CLOSED, DEGRADED, or HALF-OPEN |
| `failureCount` | Accumulated failures since last success |
| `lastFailureAt` | ISO timestamp of most recent failure |
| `retryAfterMs` | Milliseconds until next retry attempt |

Populated by `buildHealthPayload` in lines 23-44 of [`observability.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/observability.ts), with data sourced from the circuit breaker registry and `providerHealth` tracking.

### Connection Health

The **`summarizeConnectionCooldown`** function (lines 8-41) reports connection pool status per provider:

- Total active connections
- Connections in cooldown state
- Time until earliest retry

This helps diagnose provider congestion and backpressure conditions.

### Codex Account Pools

For OpenAI Codex integrations, **`summarizeCodexAccountPools`** (lines 52-87) exposes:

- Available accounts in pool
- Limited/quota-bound accounts
- Quota observations per account
- Soonest retry timestamp

### Rate Limiting and Adaptive Controls

OmniRoute live telemetry includes multiple interrelated limit systems:

- **`rateLimitStatus`** (lines 100-101) — Current global counters per provider
- **`learnedLimits`** (lines 104-105) — Adaptive thresholds derived from traffic patterns
- **`lockouts`** (lines 102-103) — Per-model lockout counts for quota and permission errors

### Session and Admission Telemetry

Active request handling surfaces through:

- **`buildSessionsSummary`** (lines 30-46) — Active session count, sticky-bound connections, top-8 sessions with request volume and age metrics
- **`projectAdaptiveAdmissionSummary`** (lines 92-118) — Runtime admission state including mode, limits, utilization percentage, pressure score, and shutdown flag
- **`projectChatAdmissionSummary`** (lines 44-61) — Structural chat admission health: heavy-lease status, waiting queue depth, byte budget consumption, and shed counts

### Additional Telemetry Fields

| Category | Source | Purpose |
|----------|--------|---------|
| **Local Provider Health** | `localProviders` (line 104) | Self-hosted LLM status |
| **Quota Monitor** | `quotaMonitorSummary` / `quotaMonitorMonitors` (lines 106-108) | Session exhaustion tracking with 8-snapshot history |
| **Credential Health** | `credentialHealth` (line 108) | API key validity totals (healthy, failed, unknown, stale) |
| **Deduplication** | `dedup.inflightRequests` (line 114) | Active request deduplication entry count |
| **Cryptography** | `cryptography.status` (lines 16-21) | Storage encryption key validity |
| **Setup Completion** | `setupComplete` (line 23) | Configuration wizard finish flag |

## Consuming Telemetry Programmatically

### Node.js Client Example

```typescript
import fetch from "node-fetch";

interface HealthPayload {
  version: string;
  buildSha: string;
  uptime: number;
  activeConnections: number;
  providerHealth: Record<string, {
    state: "OPEN" | "CLOSED" | "DEGRADED" | "HALF-OPEN";
    failureCount: number;
    lastFailureAt?: string;
  }>;
  rateLimitStatus: Record<string, number>;
  adaptiveAdmission: {
    mode: string;
    utilization: number;
    pressure: number;
    shutdown: boolean;
  };
}

async function fetchHealth(): Promise<HealthPayload> {
  const res = await fetch("http://localhost:20128/api/monitoring/health", {
    headers: process.env.ADMIN_TOKEN 
      ? { Authorization: `Bearer ${process.env.ADMIN_TOKEN}` }
      : undefined
  });
  
  if (!res.ok) throw new Error(`Health check failed: ${res.status}`);
  return res.json();
}

// Alert on degraded providers
fetchHealth().then(payload => {
  const degraded = Object.entries(payload.providerHealth)
    .filter(([, h]) => h.state === "DEGRADED" || h.state === "OPEN");
  
  if (degraded.length > 0) {
    console.warn("Provider issues detected:", degraded.map(([name]) => name));
  }
});

```

## Source Code Reference

OmniRoute live telemetry implementation spans these key files:

| File | Responsibility |
|------|---------------|
| [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts) | HTTP GET/DELETE handlers, response caching, authentication gating |
| [`src/lib/monitoring/observability.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/observability.ts) | Core aggregation functions: `buildHealthPayload`, `summarizeConnectionCooldown`, `summarizeCodexAccountPools`, `buildSessionsSummary`, `projectAdaptiveAdmissionSummary`, `projectChatAdmissionSummary` |
| [`src/lib/monitoring/buildSha.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/buildSha.ts) | Git SHA extraction for deployment tracking |
| [`src/lib/monitoring/providerHealthMatrix.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthMatrix.ts) | Provider-level dashboard matrices |
| [`src/lib/monitoring/providerHealthAutopilot.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthAutopilot.ts) | Autopilot health calculations |
| [`src/lib/monitoring/comboHealthAutopilot.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/comboHealthAutopilot.ts) | Combined health scoring for UI components |

## Summary

- **Single endpoint**: OmniRoute live telemetry consolidates 17 diagnostic categories at `/api/monitoring/health`
- **Refresh rate**: All metrics update every 1000ms with server-side caching
- **Authentication tiers**: Full payload requires admin token; reduced public view available unauthenticated
- **Operational controls**: DELETE requests reset all circuit breakers instantly
- **Key source**: [`src/lib/monitoring/observability.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/observability.ts) contains the primary aggregation logic with function-specific line ranges preserved in the codebase

## Frequently Asked Questions

### What is the refresh rate for OmniRoute live telemetry?

The telemetry payload regenerates every 1000 milliseconds (1 second) with a TTL-enforced cache. This design balances real-time observability against measurement overhead. The caching layer in [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts) prevents redundant calculation during high-frequency polling.

### How do I reset a stuck provider without restarting OmniRoute?

Send an authenticated DELETE request to `/api/monitoring/health`. This resets all circuit breakers to the CLOSED state, immediately clearing OPEN or DEGRADED conditions across every provider. Individual provider reset is not exposed; the operation is global by design.

### Why does the unauthenticated health endpoint return fewer fields?

The public view omits sensitive operational data including credential health, detailed quota monitors, and cryptographic status. This allows safe exposure to load balancers and external health checkers without leaking deployment internals. Authentication scope validation occurs in [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts).

### Can I export OmniRoute live telemetry to Prometheus or Datadog?

Not natively. The endpoint returns JSON over HTTP; integration requires a translation layer. The structured payload in `buildHealthPayload` uses consistent naming conventions suitable for metric extraction. Many operators deploy a sidecar that polls the endpoint and emits to their observability platform.