# How OmniRoute's Circuit Breaker Resilience Layer Handles Provider Failures

> Discover how OmniRoute's circuit breaker resilience layer manages provider failures. Learn about state transitions, error handling, and recovery mechanisms for robust system resilience.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-28

---

**TLDR:** OmniRoute's circuit breaker resilience layer isolates failing downstream providers using a state machine that transitions from Closed to Degraded to Open based on configurable failure thresholds, rejects requests with `CircuitBreakerOpenError` when open, and recovers through Half-Open probes, with all states persisted to SQLite and managed through a global registry with automatic cleanup.

The OmniRoute platform safeguards AI provider integrations through a sophisticated circuit breaker resilience layer that prevents cascading failures across distributed chat services. Located in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), this implementation wraps provider requests with automatic failure detection, granular threshold configuration, and durable state persistence to maintain system stability during outages.

## Core State Machine and Architecture

In [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), each provider receives a dedicated circuit breaker instance through the `getCircuitBreaker()` registry function. The implementation follows the classic **Closed → Degraded → Open → Half-Open → Closed** state machine to manage provider health.

Breakers initialize with configurable options including `failureThreshold`, `degradationThreshold`, `resetTimeout`, and `halfOpenRequests`. The constructor applies these settings while attempting to restore previous state from the SQLite `domainState` table via `_restoreFromDb()`.

## Execution Wrapper and Failure Detection

The public `execute(fn)` method serves as the primary interface for wrapping provider calls. This method first refreshes cooldown timers, then evaluates whether to proceed:

- **Immediate Rejection**: If the breaker is **OPEN** or **HALF_OPEN** with no remaining probes, `execute()` throws `CircuitBreakerOpenError` without invoking the function.
- **Execution**: Otherwise, it runs the supplied async function and routes results to `_onSuccess()` or `_onFailure()` based on the `isFailure` predicate.

When a call succeeds, `_onSuccess()` immediately closes an **OPEN** or **HALF_OPEN** circuit, or gradually decays the failure counter in **CLOSED/DEGRADED** states. On failure, `_onFailure(kind)` increments global and per-kind counters, records timestamps, and evaluates threshold breaches.

## Configurable Thresholds and Back-off Escalation

The circuit breaker resilience layer supports two tiers of failure detection:

**Kind-Specific Thresholds**: Configured via `kindThresholds`, specific failure types like `rate_limit` or `quota_exhausted` can trigger immediate opening (`immediateOpen: true`) or custom thresholds distinct from the global counter.

**Global Thresholds**: When aggregate failures reach `degradationThreshold`, the breaker transitions to **DEGRADED**. Crossing `failureThreshold` forces the circuit **OPEN**, blocking all subsequent requests until the reset timeout elapses.

The `_effectiveResetTimeout()` method calculates cooldown periods that escalate after repeated open-close cycles using `backoffEscalationCount` and `maxBackoffMultiplier`. Additionally, `cooldownByKind` allows per-failure-type timeout overrides for fine-tuned recovery semantics.

## State Persistence and Registry Management

To survive process restarts, the breaker writes state changes to SQLite through `saveCircuitBreakerState()` in the `domainState` table. The `_restoreFromDb()` method hydrates breaker status on construction, ensuring continuity across deployments.

All breakers reside in a global `Map` managed by the registry. A periodic `_registrySweep` evicts **CLOSED** breakers idle for 30 minutes, enforcing a `MAX_REGISTRY_SIZE` of 500 entries to prevent memory leaks. Consumers like [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts) import `getCircuitBreaker()` and pass provider-specific options to protect request pipelines.

## Implementation Example

The following pattern demonstrates protecting an OpenAI provider with custom thresholds:

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

const breaker = getCircuitBreaker('openai-gpt-4', {
  failureThreshold: 10,
  resetTimeout: 60_000,
  halfOpenRequests: 2,
  kindThresholds: {
    rate_limit: { threshold: 5, immediateOpen: true },
    quota_exhausted: { threshold: 3 },
  },
});

async function fetchChat(payload: any) {
  return breaker.execute(() => fetchOpenAIChat(payload));
}

```

For health monitoring, inspect breaker status across all providers:

```typescript
import { getAllCircuitBreakerStatuses } from '@/shared/utils/circuitBreaker';

export async function getProviderHealth() {
  const statuses = getAllCircuitBreakerStatuses();
  return statuses.map(s => ({
    name: s.name,
    state: s.state,
    failures: s.failureCount,
    retryAfterMs: s.retryAfterMs,
  }));
}

```

## Summary

- OmniRoute's circuit breaker resilience layer in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) implements a **Closed → Degraded → Open → Half-Open** state machine to isolate provider failures.
- The `execute()` method wraps async functions, throwing `CircuitBreakerOpenError` when circuits are open and routing outcomes to `_onSuccess()` or `_onFailure()` handlers.
- **Kind-specific thresholds** allow immediate tripping for critical failures like rate limits, while global thresholds trigger gradual degradation.
- **Exponential back-off** escalates reset timeouts through `_effectiveResetTimeout()` to prevent thundering herds during recovery.
- SQLite persistence via `domainState` ensures breaker states survive restarts, while the registry sweep caps memory usage at 500 breakers.

## Frequently Asked Questions

### How does OmniRoute's circuit breaker resilience layer differentiate between temporary and critical failures?

The implementation classifies failures through the `kindThresholds` configuration option. Critical failures like `rate_limit` can set `immediateOpen: true` to instantly open the circuit, while temporary errors respect the standard `failureThreshold` accumulation. This granular control prevents unnecessary isolation for transient network blips while immediately protecting against quota exhaustion.

### What happens to requests when a provider's circuit breaker is in the Open state?

When the breaker is **OPEN**, the `execute()` method immediately rejects all incoming requests by throwing `CircuitBreakerOpenError` without invoking the wrapped function. This fast-fail behavior prevents resource exhaustion and gives the provider time to recover before the breaker transitions to **HALF_OPEN** and allows probe requests through.

### How does the circuit breaker resilience layer maintain state across application restarts?

The breaker persists its current state to the SQLite `domainState` table via `saveCircuitBreakerState()` after every state transition. During construction, `_restoreFromDb()` attempts to load the previous state, ensuring that open circuits remain open and failure counts persist even through process restarts or deployments.

### Why does OmniRoute use a global registry with automatic cleanup for circuit breakers?

The global `Map` registry in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) ensures that multiple call sites share the same breaker instance for a given provider name, maintaining consistent failure counting. The `_registrySweep` mechanism removes idle **CLOSED** breakers after 30 minutes of inactivity and enforces a `MAX_REGISTRY_SIZE` of 500, preventing memory leaks in long-running services that might dynamically create thousands of provider configurations.