# How OmniRoute Provider Circuit Breakers Operate with Their Lazy Recovery Mechanism

> Discover how OmniRoute's provider circuit breakers use lazy recovery to refresh state only when queried. Ensure providers retry as soon as cooldown expires without background timers.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-08-06

---

**OmniRoute's circuit breakers use a lazy recovery mechanism that refreshes state only when queried, eliminating background timers while ensuring providers are retried as soon as their cooldown expires.**

OmniRoute protects its LLM routing pipeline from cascading failures by wrapping each provider with a **circuit breaker** that implements a novel **lazy recovery mechanism**. Unlike traditional implementations that spawn background timers to reset open circuits, OmniRoute's breaker in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) defers all state transitions until a caller explicitly checks the provider's status. This design reduces resource overhead while maintaining responsive failure recovery.

## Circuit Breaker State Machine and Stages

The breaker implements a five-state machine that includes a **degraded** intermediate stage for early warning:

```

CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED

```

### Normal Operation to Degraded

When a provider operates normally, the breaker remains in `CLOSED` state. As failures accumulate, the breaker enters `DEGRADED` once the failure count exceeds a configurable threshold—approximately 60% of the normal failure threshold according to the implementation in lines 12–18.

### Open and Recovery States

Once failures cross the full threshold, `_openCircuit()` transitions to `OPEN` and a cooldown period begins. The breaker will not allow requests until this period elapses. The key innovation: **no timer tracks this expiration**.

## Lazy Recovery: How It Works

The lazy recovery mechanism eliminates active polling or scheduled tasks. Instead, state expiration is evaluated **on demand** through a single private method.

### The `_refreshOpenState()` Method

```typescript
// From src/shared/utils/circuitBreaker.ts (lines 48-52)
private _refreshOpenState(): void {
  if (this._state === STATE.OPEN && this._shouldAttemptReset()) {
    this._transition(STATE.HALF_OPEN, "timeout-elapsed");
    this._halfOpenRequestsRemaining = this._halfOpenRequests;
  }
}

```

This method checks whether `_effectiveCooldown()` has elapsed **only when invoked by a status query**. If the timeout has passed, it atomically transitions to `HALF_OPEN` and initializes the probe request counter.

### Entry Points That Trigger Refresh

Three public methods invoke `_refreshOpenState()` before returning:

| Method | Purpose | Recovery Trigger |
|--------|---------|----------------|
| `canExecute()` | Permission check before routing request | Yes |
| `getStatus()` | Full state inspection for health dashboards | Yes |
| `getRetryAfterMs()` | Time-until-ready for client retry loops | Yes |

This guarantees that **any component querying a breaker sees fresh state** without dedicated scheduling infrastructure.

## Per-Kind Failure Handling and Cooldowns

OmniRoute classifies errors by **kind**—rate limits, quota exhaustion, authentication failures, and more. The implementation in lines 77–85 allows:

- **Immediate circuit open** for specific error types
- **Custom cooldown durations** per kind
- **Escalating backoff** on repeated open→probe→open cycles via `_effectiveResetTimeout()`

```typescript
// Example: kind-specific handling in failure recording
_onFailure(error: ProviderError): void {
  const kind = this._classifyError(error);
  const threshold = this._thresholds.get(kind) ?? this._defaultThreshold;
  
  // Immediate open for quota-exhausted errors
  if (kind === 'quota-exhausted') {
    this._openCircuit({ cooldownMs: this._extendedCooldown });
    return;
  }
  
  // Standard accumulation for other failures
  this._failureCounts.set(kind, (this._failureCounts.get(kind) ?? 0) + 1);
}

```

## The Recovery Cycle in Practice

### Step-by-Step Flow

1. **Failure detection** — `_onFailure()` increments counters and may call `_openCircuit()`
2. **Request blocking** — `canExecute()` returns `false` while `OPEN`, with `_refreshOpenState()` finding cooldown incomplete
3. **Elapsed cooldown detection** — first query after timeout triggers `_transition(STATE.HALF_OPEN, "timeout-elapsed")`
4. **Probe allowance** — up to `halfOpenRequests` calls succeed through `canExecute()`
5. **Resolution** — success calls `_onSuccess()` to close; failure re-opens with increased backoff

```typescript
// Typical pipeline integration
const breaker = getCircuitBreaker(providerName);

if (!breaker.canExecute()) {
  // Triggers _refreshOpenState() internally
  throw new CircuitBreakerOpenError(
    `Provider ${providerName} circuit open`,
    providerName,
    breaker.getRetryAfterMs(), // Also triggers refresh
  );
}

// Execute actual LLM call
const response = await llmClient.complete(request);
breaker.recordSuccess(); // Internal _onSuccess() handles transition

```

## Persistence and Observability

### SQLite State Survival

Breaker state persists to [`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts), enabling:

- **Process restart recovery** — open circuits remain open across deployments
- **Historical analysis** — failure patterns visible in domain-state table
- **Cross-instance coordination** — single source of truth for multi-worker deployments

### Health Endpoint Integration

The monitoring route at [`src/app/api/v1/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/monitoring/health/route.ts) exposes real-time breaker status:

```typescript
// Health dashboard integration
app.get("/api/v1/monitoring/health", async (req, res) => {
  const providers = Object.keys(providerRegistry);
  const breakerStatuses = providers.map((p) => 
    getCircuitBreaker(p).getStatus()  // Forces refresh on each call
  );
  res.json({ providers: breakerStatuses });
});

```

Operators observe lazy recovery in real time: `retryAfterMs` counts down only when the endpoint is polled, and state transitions appear on the next query after cooldown expiration.

## Comparison: Lazy vs. Eager Recovery

| Approach | Implementation | Resource Cost | Recovery Latency |
|----------|---------------|-------------|------------------|
| **Lazy (OmniRoute)** | Refresh on read | Zero idle overhead | Zero to one polling interval |
| **Eager (traditional)** | Background timer/setTimeout | Per-breaker timer | Immediate on expiration |

OmniRoute's lazy approach suits serverless and edge environments where background tasks may be frozen or throttled. The tradeoff—slightly delayed recovery if no component queries the breaker—is mitigated by omnipresent health checks and active routing engines.

## Summary

- OmniRoute circuit breakers implement **lazy recovery** through `_refreshOpenState()` invoked only on status queries
- **No background timers** run; state transitions occur in `canExecute()`, `getStatus()`, or `getRetryAfterMs()`
- The **five-state machine** includes `DEGRADED` for early intervention before full circuit open
- **Per-kind error handling** enables differentiated thresholds and cooldowns
- **SQLite persistence** in [`domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/domainState.ts) ensures survival across restarts
- **Health endpoint integration** provides observable, on-demand recovery status

## Frequently Asked Questions

### What triggers a circuit breaker to move from OPEN to HALF_OPEN?

The transition occurs when any code path calls a method that invokes `_refreshOpenState()`—typically `canExecute()`, `getStatus()`, or `getRetryAfterMs()`—and the elapsed time since opening exceeds `_effectiveCooldown()`. No timer or scheduled task performs this check; it happens exclusively on demand.

### How does lazy recovery affect provider availability after cooldown?

Recovery latency depends on query frequency. In practice, OmniRoute's routing engine checks `canExecute()` before every request attempt, and health endpoints poll `getStatus()` regularly. This ensures providers become eligible for traffic within one polling interval of cooldown expiration without dedicated timer infrastructure.

### Can circuit breaker state survive application restarts?

Yes. The implementation persists state to an SQLite table via [`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts), including last failure time, failure counts by kind, and current state. On initialization, breakers reload this state and immediately apply lazy recovery logic—an open circuit with expired cooldown will transition to `HALF_OPEN` on its first query.

### What happens when probe requests fail in HALF_OPEN state?

Failure during the `halfOpenRequests` probe window immediately re-opens the circuit. The `_onFailure()` handler applies escalating backoff through `_effectiveResetTimeout()`, which increases duration based on the count of open→probe→open cycles. This prevents rapid oscillation on persistently unhealthy providers.