# How to Debug OmniRoute Routing Issues Using the Circuit Breaker Dashboard

> Debug OmniRoute routing issues with the circuit breaker dashboard. Inspect real-time provider states, failure counts, and recovery timestamps via the /api/monitoring/health endpoint.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-08

---

**You can debug OmniRoute routing failures by querying the `/api/monitoring/health` endpoint to inspect real-time provider circuit breaker states, failure counts, and automatic recovery timestamps.**

OmniRoute implements resilient request routing through layered protection mechanisms that isolate failing providers. When upstream services return HTTP 5xx errors or time out, the system activates circuit breakers to prevent cascading failures. The **circuit breaker dashboard** exposes internal health metrics through a dedicated API endpoint, allowing you to diagnose why requests are being dropped or rerouted.

## Understanding OmniRoute's Circuit Breaker Architecture

OmniRoute protects request routing with three distinct resilience layers: **provider circuit breakers**, **connection cooldowns**, and **model lockouts**. When a provider repeatedly fails, the circuit breaker flips to **OPEN** and temporarily blocks all traffic to that provider while the routing engine falls back to healthy candidates.

The core implementation resides in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), which manages state tracking, back-off timers, and failure thresholds. According to the **Resilience Guide** in [`docs/architecture/RESILIENCE_GUIDE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/architecture/RESILIENCE_GUIDE.md), these mechanisms work together to ensure that transient outages do not degrade overall system availability.

## Accessing the Circuit Breaker Dashboard

The dashboard exposes provider health data through the internal endpoint **`/api/monitoring/health`**. The route handler is implemented in [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts), which serializes the in-memory breaker states into a JSON payload.

When you query this endpoint, the response contains an array of provider objects with the following diagnostic fields:

- **`provider`**: Human-readable identifier (e.g., `openai`, `anthropic`).
- **`circuitBreakerState`**: Current state as `"CLOSED"`, `"OPEN"`, or `"HALF_OPEN"`.
- **`circuitOpenUntil`**: Epoch-millisecond timestamp indicating when the **OPEN** state expires.
- **`failureCount`**: Consecutive failures that triggered the current state.
- **`resetTimeoutMs`**: Configured back-off timeout for this provider.
- **`lastError`**: The most recent error message (e.g., "504 Gateway Timeout").

## Interpreting Dashboard Metrics

### Circuit Breaker States

The dashboard reports three distinct states that indicate provider health:

- **CLOSED**: The provider is healthy and accepting traffic. If routing issues persist while the state is closed, investigate rate limits or model lockouts instead.
- **OPEN**: The provider is blocked due to consecutive failures. Check `circuitOpenUntil` to determine when automatic recovery will occur.
- **HALF_OPEN**: The system is probing the provider with a test request. A successful probe closes the circuit; a failure reopens it.

### Key Diagnostic Fields

The **`lastError`** field reveals the root cause of the failure streak, such as "502 Bad Gateway" or connection timeouts. Correlate this with **`failureCount`** to determine if the provider is approaching its threshold (OAuth providers default to 3 failures, API-key providers to 5, as defined in [`src/shared/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/config/constants.ts)).

The **`circuitOpenUntil`** timestamp implements lazy recovery, ensuring the provider remains isolated until the configured `resetTimeoutMs` elapses.

## Step-by-Step Debugging Workflow

Follow this systematic approach to diagnose routing anomalies using the dashboard:

1. **Query the health endpoint** using `curl http://localhost:20128/api/monitoring/health` or fetch it from your browser console.

2. **Locate the target provider** in the JSON response array.

3. **Check `circuitBreakerState`**:
   - If `CLOSED`, examine `lastError` and request logs for non-circuit issues.
   - If `OPEN`, note the `circuitOpenUntil` timestamp to estimate recovery time.
   - If `HALF_OPEN`, wait for the probe to complete or investigate immediate failures.

4. **Inspect `lastError`** to identify whether the issue stems from network timeouts, HTTP errors, or authentication failures.

5. **Correlate with request logs** in [`open-sse/utils/requestLogger.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/requestLogger.ts) to confirm the request was short-circuited before reaching the upstream.

6. **Adjust configuration** if thresholds are too aggressive by modifying environment variables or constants in [`src/shared/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/config/constants.ts).

## Manual Reset and Configuration Tuning

If a circuit breaker is open but you have verified the provider is healthy, you can force recovery by invoking **`resetAllCircuitBreakers()`** from [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts). This function clears the in-memory state immediately, though the breaker will reopen if failures persist.

Provider-specific thresholds reside in [`src/shared/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/config/constants.ts), where OAuth providers default to 3 consecutive failures and API-key providers default to 5. Override these values via environment variables to tune sensitivity based on your reliability requirements.

## Practical Code Examples

Fetch the dashboard programmatically using Node.js:

```javascript
import fetch from 'node-fetch';

async function getCircuitStatus() {
  const res = await fetch('http://localhost:20128/api/monitoring/health');
  const data = await res.json();

  data.providers.forEach(p => {
    console.log(
      `${p.provider}: ${p.circuitBreakerState}` +
      (p.circuitBreakerState === 'OPEN' ? ` (unblocks @ ${new Date(p.circuitOpenUntil)})` : '')
    );
  });
}

getCircuitStatus();

```

Quick command-line inspection with curl and jq:

```bash
curl -s http://localhost:20128/api/monitoring/health | jq .

```

Render provider status in a React component:

```tsx
import useSWR from 'swr';

export function ProviderBadge({ provider }: { provider: string }) {
  const { data } = useSWR('/api/monitoring/health', url => fetch(url).then(r => r.json()));
  const info = data?.providers?.find(p => p.provider === provider);

  if (!info) return null;
  const color = info.circuitBreakerState === 'OPEN' ? 'red' : 'green';
  return <span style={{ color }}>{info.circuitBreakerState}</span>;
}

```

## Summary

- Query `/api/monitoring/health` (implemented in [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts)) to retrieve real-time circuit breaker states for all providers.
- Interpret `circuitBreakerState` values (`CLOSED`, `OPEN`, `HALF_OPEN`) to determine if routing issues stem from circuit isolation or other factors.
- Use `circuitOpenUntil` and `lastError` to diagnose recovery timelines and root causes.
- Invoke `resetAllCircuitBreakers()` from [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) for manual recovery after verifying provider health.
- Adjust failure thresholds in [`src/shared/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/config/constants.ts) to balance resilience against sensitivity.

## Frequently Asked Questions

### How do I know if a routing failure is caused by a circuit breaker or a rate limit?

Check the `circuitBreakerState` field in the health endpoint response. If the state is `CLOSED`, the provider is healthy and the failure likely stems from rate limiting or model lockouts. If the state is `OPEN` or `HALF_OPEN`, the circuit breaker has isolated the provider due to consecutive errors, which you can verify by inspecting the `lastError` field for HTTP 5xx or timeout messages.

### What is the difference between HALF_OPEN and OPEN states?

An `OPEN` circuit completely blocks traffic to the provider until the `circuitOpenUntil` timestamp passes. `HALF_OPEN` indicates the system is testing the provider with a single probe request. If the probe succeeds, the state returns to `CLOSED`; if it fails, the circuit reverts to `OPEN` with a refreshed timeout. This allows automatic recovery without manual intervention.

### Can I manually close a circuit breaker before the timeout expires?

Yes. You can manually reset the circuit by calling `resetAllCircuitBreakers()` from [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), which clears the in-memory state immediately. However, only use this after confirming the provider is genuinely healthy, as premature reset will cause the breaker to reopen quickly if failures persist.

### Where are the circuit breaker thresholds configured?

Default failure thresholds are defined in [`src/shared/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/config/constants.ts), with OAuth providers typically set to 3 failures and API-key providers to 5. You can override these values through environment variables to adjust how quickly the system isolates unreliable providers based on your specific reliability requirements.