# How OmniRoute's Circuit Breaker Prevents Cascading Provider Failures

> Learn how OmniRoute's circuit breaker design prevents cascading provider failures. It isolates failing providers, returning immediate errors to protect your system.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-03

---

**OmniRoute isolates failing upstream providers by wrapping every API call in a per-provider circuit breaker that tracks error counts and opens when configurable thresholds are exceeded, immediately returning a `CircuitBreakerOpenError` instead of hitting the upstream to stop cascading failures through the combo routing pipeline.**

OmniRoute, an open-source multi-provider routing engine, implements a defensive resilience layer to prevent a single misbehaving AI provider from triggering retry storms and resource exhaustion across the entire request pipeline. The circuit breaker pattern, implemented in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), maintains independent state machines for each provider endpoint to ensure localized failure containment. This architecture guarantees that HTTP errors like 502, 429, or 5xx responses from one provider do not propagate through the combo routing logic and degrade the entire system.

## Per-Provider Isolation Architecture

OmniRoute’s circuit breaker guarantees that only the failing provider is blocked, not the entire routing combo or other healthy upstreams.

### The In-Memory Registry (`getCircuitBreaker`)

Each provider receives its own `CircuitBreaker` instance via the `getCircuitBreaker(name)` function, which stores instances in an in-memory registry defined in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts). This registry maps provider identifiers to specific breaker objects, ensuring complete isolation between different API endpoints. When the combo router iterates through multiple targets, it retrieves a unique breaker for each provider using `getCircuitBreaker(`provider:${providerId}`)`, preventing cross-provider state contamination.

### Persistent State Across Restarts (`saveCircuitBreakerState`)

To prevent a failing provider from being silently retried after a server crash or deployment, the breaker persists its state to SQLite using `saveCircuitBreakerState` and `loadCircuitBreakerState`. The internal methods `_persistToDb()` and `_restoreFromDb()` serialize the current state (OPEN, CLOSED, etc.) and failure counts to the database layer in [`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts). This persistence ensures that an open circuit remains open across process restarts, maintaining protection continuity without manual operator intervention.

## State Machine and Failure Detection

The circuit breaker operates as a deterministic state machine transitioning between **CLOSED**, **DEGRADED**, **OPEN**, **HALF_OPEN**, and back to **CLOSED**, with specific triggers for each transition.

### CLOSED to DEGRADED to OPEN Transitions

In the `CLOSED` state, the breaker forwards all requests normally while tracking failures. When the failure count reaches **60%** of the configured threshold, the breaker transitions to `DEGRADED`—still forwarding requests but emitting warning logs for early visibility. Once errors exceed the full `failureThreshold`, the breaker `OPEN`s, and all subsequent calls immediately short-circuit with a `CircuitBreakerOpenError` without ever reaching the network layer. This transition logic is handled in the `_onFailure()` method within the core implementation file.

### Failure-Kind Aware Thresholds (`kindThresholds`)

Not all HTTP errors are treated equally. The implementation supports granular failure classification via `kindThresholds` and `cooldownByKind` maps. For example, rate-limit responses (429) can trigger an immediate open with `immediateOpen: true`, while transient 500 errors might tolerate higher retry counts. This discrimination prevents overly aggressive breaker opening on recoverable glitches while reacting instantly to quota exhaustion or authentication failures.

## Recovery and Back-Off Strategies

Automatic recovery mechanisms ensure that providers are not blacklisted indefinitely and that flaky endpoints do not waste resources with rapid retry attempts.

### Exponential Back-Off (`_effectiveResetTimeout`)

After each failed recovery attempt (OPEN → HALF_OPEN → OPEN), the reset timeout multiplies exponentially up to a `maxBackoffMultiplier`. The `_effectiveResetTimeout()` method calculates this dynamic delay, ensuring that repeatedly failing providers face progressively longer cooldowns (e.g., 30s → 60s → 120s) rather than fixed intervals. This back-off strategy protects the routing pipeline from wasting cycles on unstable upstreams.

### Half-Open Probing (`halfOpenRequests`)

When the reset timeout expires, the breaker enters `HALF_OPEN` state and allows a limited number of probe requests—configured via `halfOpenRequests`—to test provider recovery. The `execute()` method permits these probes while `_refreshOpenState()` monitors their success rate. If probes succeed, the breaker closes; if they fail, it reopens immediately. This controlled probing enables fast recovery detection without flooding the recovering provider with full traffic volume.

### Registry Eviction for Memory Management

To prevent memory bloat in long-running instances, the registry sweeps idle breakers every 30 minutes via `evictColdBreakersIfNeeded()`. Breakers in `CLOSED` state with no recent activity are removed from the in-memory map, keeping the registry size bounded while preserving active or open circuits. This cleanup runs automatically and requires no manual configuration.

## Integration with the Combo Router

The combo routing engine in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) leverages the breaker to implement the short-circuit pattern. Before attempting any provider call, the router checks `breaker.canExecute()`:

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

async function routeCombo(request: Request, combo: ComboConfig) {
  for (const target of combo.targets) {
    const breaker = getCircuitBreaker(`provider:${target.providerId}`);
    
    if (!breaker.canExecute()) {
      continue; // Skip failing provider immediately
    }

    try {
      return await breaker.execute(async () => {
        const resp = await fetch(target.url, { body: request.body });
        if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
        return resp.json();
      });
    } catch (err) {
      // Error already recorded by breaker; try next target
    }
  }
  throw new Error('All combo targets failed');
}

```

When `canExecute()` returns false, the router skips to the next provider in the combo strategy, effectively isolating the failure. The `execute()` wrapper automatically records success or failure, updating the state machine without manual instrumentation.

## Configuration and Usage Examples

Creating a breaker with custom thresholds for OAuth-sensitive providers:

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

const breaker = getCircuitBreaker(`provider:${providerId}`, {
  failureThreshold: 5,
  resetTimeout: 30_000,
  halfOpenRequests: 1,
  kindThresholds: {
    rate_limit: { threshold: 1, immediateOpen: true },
    quota_exhausted: { threshold: 1, immediateOpen: true }
  },
  maxBackoffMultiplier: 5
});

```

Checking breaker status before manual calls:

```typescript
if (!breaker.canExecute()) {
  throw new CircuitBreakerOpenError(
    `Provider ${providerId} is circuit-open`,
    providerId,
    breaker.getRetryAfterMs()
  );
}

```

Administrative reset for testing or manual recovery:

```typescript
import { resetAllCircuitBreakers } from '@/shared/utils/circuitBreaker';

// Clears all open states and DB entries
resetAllCircuitBreakers();

```

## Summary

- **Per-provider isolation** via `getCircuitBreaker()` ensures failures stay contained to individual upstreams.
- **State persistence** through `_persistToDb()` and `_restoreFromDb()` maintains protection across server restarts.
- **Graduated state machine** (CLOSED → DEGRADED → OPEN) provides early warnings before full isolation.
- **Kind-aware thresholds** allow aggressive opening on critical errors like rate limits while tolerating transient failures.
- **Exponential back-off** in `_effectiveResetTimeout()` prevents resource waste on repeatedly flaky providers.
- **Half-open probing** via `halfOpenRequests` enables safe recovery detection without traffic flooding.
- **Automatic eviction** via `evictColdBreakersIfNeeded()` prevents memory leaks in the breaker registry.

## Frequently Asked Questions

### What is a circuit breaker in OmniRoute?

A circuit breaker in OmniRoute is a per-provider state machine that monitors failure rates and opens when error thresholds are exceeded, immediately returning errors instead of attempting upstream requests. This pattern prevents cascading failures by isolating unhealthy providers from the routing combo, protecting system resources and maintaining overall request latency.

### How does OmniRoute handle circuit breaker state after a server restart?

OmniRoute persists circuit breaker states to SQLite using `saveCircuitBreakerState` and `loadCircuitBreakerState` functions, ensuring that an open circuit remains open after a crash or deployment. The internal `_persistToDb()` method serializes state changes to the database, while `_restoreFromDb()` hydrates the breaker registry on startup, maintaining continuity of protection without manual reset.

### What happens when a provider circuit breaker opens?

When a breaker opens, the `canExecute()` method returns false, causing the combo router to skip that provider entirely and attempt the next target in the sequence. Any direct calls to `execute()` while open immediately throw a `CircuitBreakerOpenError` without making an HTTP request, eliminating network latency and preventing load on the failing upstream.

### How does the combo router handle multiple provider failures?

The combo router iterates through its target list, checking `breaker.canExecute()` for each provider before attempting the request. If a provider is open, the router short-circuits that iteration and moves to the next healthy target. This allows the request to fail over through multiple providers until finding a healthy one, or exhausting all options, ensuring maximum availability despite individual upstream failures.