# How Circuit Breakers Are Implemented in OmniRoute: A Deep Dive into the Resilience Architecture

> Discover how OmniRoute implements circuit breakers using a four-state FSM at the provider level. Learn to configure thresholds and isolate failing services for robust resilience.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-09-01

---

**OmniRoute implements circuit breakers at the provider level through a four-state finite state machine in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), with thresholds configurable per provider type to isolate failing upstream services before they cascade through the routing layer.**

The **circuit breaker pattern** in OmniRoute is a critical resilience mechanism that prevents the AI gateway from repeatedly calling unhealthy LLM providers. Unlike simple timeout-based retries, this implementation uses a sophisticated state machine with an early-warning **DEGRADED** state to give operators visibility before full isolation occurs. The pattern is integrated directly into the server-sent event (SSE) request handlers and exposed through a health monitoring API.

## How the OmniRoute Circuit Breaker State Machine Works

The core implementation lives in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) and defines four distinct states that control traffic flow to each provider:

| State | Behavior | Transition Trigger |
|-------|----------|-------------------|
| **CLOSED** | Normal operation; all requests allowed | Accumulated failures reach degradation or open threshold |
| **DEGRADED** | Traffic continues; UI surfaces warning | Failure count exceeds `circuitBreakerDegrade` but below open threshold |
| **OPEN** | Provider blocked; requests rejected or skipped | Consecutive failures hit `circuitBreakerThreshold` |
| **HALF_OPEN** | Single probe request allowed to test recovery | Reset timeout expires automatically |

State transitions happen lazily. When `canExecute()` is called, the breaker checks `Date.now()` against the last failure timestamp plus the reset timeout. If an **OPEN** breaker has exceeded its cooldown, it automatically transitions to **HALF_OPEN** without requiring external coordination.

## Configuration Parameters and Provider Profiles

Breaker thresholds are defined in [`src/shared/constants/providers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providers.ts) and exposed through environment variables with the prefix `OMNIROUTE_PROVIDER_BREAKER_*`:

| Parameter | Environment Variable | Purpose |
|-----------|---------------------|---------|
| `circuitBreakerThreshold` | `OMNIROUTE_PROVIDER_BREAKER_THRESHOLD` | Consecutive failures before opening the breaker |
| `circuitBreakerReset` | `OMNIROUTE_PROVIDER_BREAKER_RESET` | Seconds until **OPEN** → **HALF_OPEN** transition |
| `circuitBreakerDegrade` | `OMNIROUTE_PROVIDER_BREAKER_DEGRADE` | Failure count that triggers **DEGRADED** state |

Default profiles vary by provider authentication type. OAuth providers like OpenAI use stricter defaults (8 failures, 60s reset), while local or API-key providers may tolerate more failures before degradation.

```typescript
// src/shared/constants/providers.ts (conceptual structure)
export interface ProviderCircuitConfig {
  circuitBreakerThreshold: number;
  circuitBreakerReset: number;
  circuitBreakerDegrade: number;
}

export const DEFAULT_OAUTH_PROVIDER: ProviderCircuitConfig = {
  circuitBreakerThreshold: 8,
  circuitBreakerReset: 60,
  circuitBreakerDegrade: 4,
};

```

## Integrating Circuit Breakers in Request Handlers

The breaker is consulted on every routed request through the `getCircuitBreaker()` factory. In [`src/sse/handlers/chatHelpers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chatHelpers.ts), the handler obtains a breaker instance scoped to the target provider:

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';
import { ProviderName } from '@/shared/constants/providers';

// Executed per incoming chat request
const breaker = getCircuitBreaker(ProviderName.OPENAI);

if (!breaker.canExecute()) {
  // Provider is OPEN or in cooldown - routing logic skips to fallback
  // Returns standardized error: 'provider-circuit-open'
  return { error: 'provider-circuit-open', status: 503 };
}

// Proceed with upstream request

```

After the upstream responds, the handler updates breaker state:

```typescript
// Successful response path
breaker.recordSuccess();  // Resets failure accumulator to zero

// Failure path - only server errors count toward breaker
const SERVER_ERROR_CODES = new Set([408, 500, 502, 503, 504]);

if (SERVER_ERROR_CODES.has(response.status)) {
  breaker.recordFailure();  // May trigger DEGRADED or OPEN
}

```

## What Errors Actually Trip the Circuit Breaker

Not all failures are created equal in OmniRoute's implementation. The breaker **only** reacts to server-side transport errors:

- **Trips the breaker:** `408 Request Timeout`, `500 Internal Server Error`, `502 Bad Gateway`, `503 Service Unavailable`, `504 Gateway Timeout`

- **Ignored by breaker:** `401 Unauthorized`, `403 Forbidden`, `429 Too Many Requests`

Authentication failures and rate limits are handled through separate mechanisms—**connection cooldowns** for 429s and **model lockouts** for quota exhaustion. This separation prevents transient user configuration errors from unnecessarily isolating entire providers.

## Monitoring Circuit Breaker State at Runtime

The breaker exposes internal state through [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts), enabling real-time observability:

```typescript
// Aggregate breaker status across all configured providers
const healthStatus = {
  providers: Object.values(ProviderName).map(name => {
    const breaker = getCircuitBreaker(name);
    return {
      name,
      state: breaker.getState(),        // 'CLOSED' | 'DEGRADED' | 'OPEN' | 'HALF_OPEN'
      failures: breaker.getFailureCount(),
      lastFailure: breaker.getLastFailureTime(),
      nextRetry: breaker.getNextRetryTime(),
    };
  })
};

```

This endpoint drives operational dashboards and can trigger pager duty alerts when critical providers enter **DEGRADED** or **OPEN** states.

## Summary

- **Core Location:** The `CircuitBreaker` class in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) implements a four-state machine with lazy timeout-driven transitions.

- **Configuration:** Three environment-controlled parameters (`circuitBreakerThreshold`, `circuitBreakerReset`, `circuitBreakerDegrade`) define provider-specific resilience profiles in [`src/shared/constants/providers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providers.ts).

- **Integration Point:** Request handlers in [`src/sse/handlers/chatHelpers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chatHelpers.ts) obtain breakers via `getCircuitBreaker()` and consult `canExecute()` before upstream calls.

- **Selective Failure Handling:** Only HTTP `408`, `500`, `502`, `503`, `504` errors increment the breaker; auth and rate-limit errors use separate mitigation paths.

- **Observability:** Runtime state is queryable through the health monitoring API at [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts).

## Frequently Asked Questions

### How does OmniRoute's circuit breaker differ from a simple retry loop with exponential backoff?

OmniRoute's breaker is **stateful and provider-scoped**, not request-scoped. While exponential backoff retries a single request multiple times, the circuit breaker **blocks all traffic** to a provider after threshold failures, preventing wasted resources and cascading latency. The **DEGRADED** state provides early warning without full isolation—something pure retry logic cannot offer.

### Can I disable the circuit breaker for specific providers?

Yes. Set `circuitBreakerThreshold` to `Infinity` or a sufficiently high value in your provider configuration within [`src/shared/constants/providers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providers.ts). The breaker will still track failures but never transition to **OPEN**. Note that this removes protection against upstream outages.

### What happens to requests when a provider's circuit breaker is OPEN?

Requests receive a structured error response with code `provider-circuit-open` and HTTP status `503`. The routing layer may then apply **combo routing** logic to failover to alternative providers, or return the error to the client depending on configuration.

### How is the HALF_OPEN state tested without affecting production traffic?

When timeout expires, `canExecute()` permits **exactly one request** through. Success via `recordSuccess()` closes the breaker immediately. Failure via `recordFailure()` re-opens it for another full reset period. This probe mechanism requires no separate "canary" infrastructure—it piggybacks on real user traffic with automatic rollback.