# How OmniRoute's Provider Circuit Breaker Works: A Deep Dive into the Implementation

> Discover how OmniRoute's provider circuit breaker protects your LLM routing. Learn its implementation details for automatic degradation, opening, and recovery based on failure thresholds.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-13

---

**OmniRoute protects your LLM routing system from provider failures by wrapping every upstream request in a stateful `CircuitBreaker` instance that automatically degrades, opens, and recovers based on configurable failure thresholds.**

The circuit breaker lives in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) and extends the classic pattern with failure-kind awareness, adaptive back-off, and persistent state recovery. This guide examines the four-state implementation, configuration options, and how the breaker integrates with OmniRoute's multi-provider routing logic.

---

## Four-State Circuit Breaker Design

Unlike traditional three-state breakers, OmniRoute uses **four distinct states** to provide finer-grained control over provider health:

| State | Behavior |
|-------|----------|
| **CLOSED** | Traffic passes normally; success decrements failure counters. |
| **DEGRADED** | Failure rate elevated but below threshold; requests proceed with logged warnings. |
| **OPEN** | All requests short-circuit immediately; no upstream calls made. |
| **HALF_OPEN** | Probe window allows limited requests to test provider recovery. |

The `STATE` enum is defined at `src/shared/utils/circuitBreaker.ts#L12-L16`. This four-state design lets operators monitor when a provider is "limping" before it fully fails.

---

## Failure-Kind Awareness and Classification

OmniRoute's breaker goes beyond simple success/failure counting. It classifies errors into **per-kind categories** with independent thresholds:

```typescript
// From src/shared/utils/circuitBreaker.ts#L77-84
interface KindThresholds {
  [kind: string]: {
    threshold: number;      // failures before this kind trips the breaker
    resetTimeout?: number;  // optional override for this kind's cooldown
    immediateOpen?: boolean; // true = bypass DEGRADED, jump straight to OPEN
  };
}

```

Common failure kinds include:

- **`rate_limit`** (HTTP 429 with retry-after headers)
- **`quota_exhausted`** (hard limit reached)
- **`transient`** (5xx server errors, timeouts)

This classification lets you treat a flaky 5xx differently from a quota exhaustion. For example, `immediateOpen: true` on `quota_exhausted` disables a provider instantly rather than waiting for five failures.

---

## State Transitions and Thresholds

The breaker uses **two thresholds** to drive state changes:

1. **`degradationThreshold`** — defaults to 60% of `failureThreshold`
2. **`failureThreshold`** — absolute limit before circuit opens

When a failure occurs, the logic at `src/shared/utils/circuitBreaker.ts#L79-L104`:

1. Increments global and per-kind failure counters
2. Compares against `degradationThreshold` → move to **DEGRADED**
3. Compares against `failureThreshold` → move to **OPEN** (or check `immediateOpen` flag)

On success in **HALF_OPEN**, the breaker resets to **CLOSED** and clears all counters. Success in other states gradually decrements failure counts via proportional reduction.

---

## Adaptive Back-Off and Lazy Recovery

To prevent **flapping** (rapid OPEN/CLOSED oscillation), OmniRoute implements exponential back-off:

```typescript
// From src/shared/utils/circuitBreaker.ts#L55-63
private getEffectiveResetTimeout(): number {
  const multiplier = Math.min(
    this.backoffMultiplier,
    this.options.maxBackoffMultiplier ?? 5
  );
  return this.options.resetTimeout * Math.pow(2, multiplier - 1);
}

```

Each full cycle of **OPEN → HALF_OPEN → OPEN** doubles the reset timeout up to the configured maximum.

The breaker uses **lazy recovery** — no background timers run. Instead, `canExecute()` checks `_refreshOpenState()` at `src/shared/utils/circuitBreaker.ts#L63-L66`, comparing `Date.now()` against the stored open timestamp. This eliminates timer management overhead and works correctly across process restarts.

---

## Persistence Across Restarts

Circuit breaker state survives process restarts through the **`domainState` database table** ([`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts)):

| Stored Data | Purpose |
|-------------|---------|
| `state` | Current breaker state (serialized enum) |
| `counters` | Global and per-kind failure counts |
| `options` | Configuration snapshot |
| `openedAt` / `halfOpenedAt` | Timestamps for cooldown calculation |
| `backoffMultiplier` | Current back-off level |

On construction, each breaker loads prior state via `loadCircuitBreakerState()` and restores its position in the state machine. This prevents a restart from "forgiving" a misbehaving provider.

---

## Registry Management and Memory Boundaries

Breakers are cached in a **global `Map`** keyed by provider name. To prevent unbounded growth:

- **Maximum registry size**: 500 active breakers (`MAX_REGISTRY_SIZE`)
- **Periodic sweep**: Idle breakers in **CLOSED** state are evicted
- **Factory pattern**: `getCircuitBreaker(name, options)` retrieves existing or creates new

The registry sweep runs at `src/shared/utils/circuitBreaker.ts#L18-L33`.

---

## Practical Usage Example

Here's how to integrate the breaker into a provider handler:

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

// Configure provider-specific thresholds
const anthropicBreaker = getCircuitBreaker('anthropic', {
  failureThreshold: 5,
  resetTimeout: 30_000,
  halfOpenRequests: 1,
  classifyError: (err) => {
    if (err.status === 529) return 'rate_limit';
    if (err.status >= 500) return 'transient';
    return undefined;
  },
});

// Wrap upstream call
async function callAnthropic(messages: Message[]) {
  return anthropicBreaker.execute(async () => {
    const response = await fetch('https://api.anthropic.com/v1/messages', {
      method: 'POST',
      headers: { 'x-api-key': process.env.ANTHROPIC_KEY },
      body: JSON.stringify({ messages, model: 'claude-3-opus-20240229' }),
    });
    if (!response.ok) throw await response.json();
    return response.json();
  });
}

// Pre-flight check for routing decisions
function canRouteToAnthropic(): boolean {
  return anthropicBreaker.canExecute();
}

```

The `execute()` method handles all state machine logic, error classification, and persistence automatically.

---

## Integration with OmniRoute's Fallback System

In [`src/sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/accountFallback.ts), the breaker drives provider selection:

```typescript
// Simplified excerpt from account fallback logic
if (!breaker.canExecute()) {
  // Skip this provider key, try next in rotation
  return tryNextProvider();
}

```

When a breaker is **OPEN**, the combo router immediately excludes that provider from consideration. When **HALF_OPEN**, only one concurrent probe request is permitted — subsequent calls fail fast until the probe completes.

---

## Summary

- **Four states** (CLOSED, DEGRADED, OPEN, HALF_OPEN) provide granular health visibility
- **Per-failure-kind thresholds** let you customize behavior for rate limits, quotas, and transient errors
- **Adaptive back-off** with exponential increase prevents flapping on flaky providers
- **Lazy recovery** eliminates timer overhead while maintaining accurate cooldowns
- **Database persistence** ensures protection survives process restarts
- **Registry bounds** keep memory usage predictable with automatic cleanup

---

## Frequently Asked Questions

### How does OmniRoute's circuit breaker differ from standard implementations?

Standard breakers use three states (CLOSED, OPEN, HALF_OPEN). OmniRoute adds **DEGRADED** as an intermediate warning state, plus **failure-kind classification** that lets rate limits, quota errors, and transient failures have independent thresholds and behaviors. The adaptive back-off and database persistence are also atypical enhancements.

### What happens when a provider circuit opens mid-request?

The breaker checks `canExecute()` **before** any upstream call. If already OPEN, the request aborts immediately with a standardized error that the routing layer interprets as "skip this provider." No network I/O occurs, and the combo router selects an alternative provider from its pool.

### Can I disable circuit breaking for specific providers?

Yes — pass `failureThreshold: Infinity` or a very high number in `CircuitBreakerOptions`. However, this removes protection against cascading failures. A better practice is setting `immediateOpen: false` for specific failure kinds while keeping the overall breaker active.

### How do I monitor circuit breaker health?

Call `breaker.getStatus()` to retrieve:

```typescript
{
  state: 'OPEN' | 'CLOSED' | 'DEGRADED' | 'HALF_OPEN',
  failureCount: number,
  retryAfterMs: number,  // 0 if canExecute() would return true
  lastFailureKind?: string
}

```

Integrate these metrics into your observability stack to detect provider degradation before users experience errors.