# Resilience Layers in OmniRoute: Circuit Breakers, Connection Cooldown, and Model Lockout

> Explore OmniRoute's three resilience layers: circuit breakers, connection cooldown, and model lockout. Prevent cascading failures and isolate flaky providers.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-07-04

---

**OmniRoute implements a three-layer resilience system—provider-wide circuit breakers, per-failure-kind connection cooldown, and model-level lockout—to isolate flaky upstream providers and prevent cascading failures across the routing pipeline.**

The `diegosouzapw/OmniRoute` repository protects its AI model routing infrastructure through hierarchical fault tolerance mechanisms. These resilience layers guard against systemic outages, rate-limit spikes, and individual model malfunctions without requiring manual intervention. This article examines the implementation details, configuration options, and interaction patterns between these safeguards.

## The Three-Layer Resilience Architecture

OmniRoute's resilience strategy operates at three distinct granularities: the entire provider, specific failure types, and individual model connections.

### Provider-Wide Circuit Breaker

The first layer sits at [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), where the `CircuitBreaker` class monitors all requests targeting a specific provider (such as OpenAI or Gemini).

**State Machine Implementation**

The breaker maintains a five-state lifecycle: **CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED**. 

- In the **CLOSED** state, requests flow normally while the system tracks consecutive failures
- After exceeding the `failureThreshold` (default 5), the breaker transitions to **OPEN**, short-circuiting all subsequent calls
- Following the `resetTimeout` period, the state shifts to **HALF_OPEN**, allowing a limited number of probe requests (`halfOpenRequests`)
- Successful probes close the circuit; failed probes reopen it

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

// Configure a breaker for the "openai" provider
const breaker = getCircuitBreaker('openai', {
  failureThreshold: 5,
  resetTimeout: 30_000,      // 30 seconds default cooldown
  halfOpenRequests: 1,
});

async function callOpenAI(payload: unknown) {
  if (!breaker.canExecute()) {
    throw new Error('OpenAI is currently unavailable (circuit open)');
  }

  return breaker.execute(async () => {
    // HTTP request logic here
    return await fetchProvider(payload);
  });
}

```

### Connection Cooldown (Per-Failure-Kind Back-Off)

The second layer addresses specific failure classifications through the `cooldownByKind` map within the same [`circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/circuitBreaker.ts) file. This mechanism overrides the generic `resetTimeout` when the provider returns specific error types like rate limits or quota exhaustion.

**Failure Classification Logic**

The `classifyError` function categorizes failures, allowing the breaker to apply targeted back-off strategies. When a 429 (rate limit) or quota-exhausted error occurs, the system uses the `_effectiveCooldown` calculation to select the most restrictive timeout available for that specific failure kind.

```typescript
const breaker = getCircuitBreaker('openai', {
  failureThreshold: 5,
  resetTimeout: 30_000,
  // Override cooldowns for specific failure types
  cooldownByKind: { 
    rate_limit: 120_000,      // 2 minutes for rate limits
    quota_exhausted: 300_000  // 5 minutes for quota issues
  },
});

```

This prevents "hammering" a provider during transient rejection periods while allowing faster recovery for other error types.

### Model-Level Lockout

The third layer isolates individual model-connection pairs through [`src/domain/lockoutPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/lockoutPolicy.ts). While the circuit breaker handles provider-wide health, the lockout policy tracks specific model instances (such as *Gemini-1.5-flash* on a particular account).

**Lockout Mechanism**

The `LockoutPolicy` class counts failures per identifier using a sliding window (`attemptWindowMs`). When failures exceed `maxAttempts` within this window, the system persists a lockout record for `lockoutDurationMs`. During lockout, the model is excluded from combo resolution and selection algorithms.

The [`src/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/services/accountFallback.ts) service consumes this policy through `markAccountUnavailable()`, which triggers when specific models return repeated 429 or 404 errors.

```typescript
import { markAccountUnavailable } from '@/services/accountFallback';

// Trigger lockout for a specific model returning rate limits
await markAccountUnavailable({
  provider: 'gemini',
  connectionId: 'conn-123',
  model: 'gemini-1.5-flash',
  reason: 'rate_limit',
});

// The combo resolver automatically skips locked models
const combo = await resolveComboTargets(...);

```

## How the Layers Interact

These resilience mechanisms operate hierarchically during request routing:

1. **Circuit-Breaker Evaluation**: The system first checks if the provider's circuit breaker is **OPEN**. If so, the provider is excluded from combo resolution entirely.

2. **Cooldown Application**: When the breaker is **CLOSED** or **DEGRADED**, the connection cooldown layer may still apply delays based on the last failure kind detected. The `cooldownByKind` configuration determines the back-off duration before the next request attempt.

3. **Lockout Verification**: Even with a healthy provider circuit, the model-level lockout layer can temporarily blacklist specific model-connection pairs. The `domain_lockout_state` table persists these records, consulted by the account-fallback logic in [`src/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/services/accountFallback.ts).

4. **Success Recovery**: A successful request through `recordSuccess()` or an explicit `forceUnlock` clears the model lockout, while successful probe requests in **HALF_OPEN** state restore the circuit breaker to **CLOSED**.

## Monitoring Resilience State

OmniRoute exposes the current state of all resilience layers for observability and dashboarding:

```typescript
import { getAllCircuitBreakerStatuses } from '@/shared/utils/circuitBreaker';
import { getAllModelLockouts } from '@/domain/lockoutPolicy';

// Current circuit states for all providers
console.log('Provider breakers:', getAllCircuitBreakerStatuses());

// Active model lockouts
console.log('Model lockouts:', await getAllModelLockouts());

```

These helper functions, located at the end of [`circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/circuitBreaker.ts) and exported from [`lockoutPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/lockoutPolicy.ts), enable real-time monitoring of which providers and models are currently isolated from the routing pool.

## Summary

- **Provider-Wide Circuit Breaker**: Located in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), implements a five-state machine (CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED) with configurable `failureThreshold` and `resetTimeout` to protect against systemic provider outages.

- **Connection Cooldown**: Implemented within the same circuit breaker file, uses the `cooldownByKind` map to apply specific back-off durations for different failure types (rate limits, quota exhaustion), preventing request hammering during transient errors.

- **Model-Level Lockout**: Defined in [`src/domain/lockoutPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/lockoutPolicy.ts) and consumed by [`src/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/services/accountFallback.ts), tracks failures per model-connection pair using sliding windows (`attemptWindowMs`) and temporarily excludes repeat offenders via persistent lockout records.

## Frequently Asked Questions

### How does OmniRoute determine when to open a circuit breaker?

The circuit breaker opens when consecutive failures exceed the `failureThreshold` parameter (default 5). According to the implementation in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), the breaker tracks failure counts through the `recordFailure()` method. When the threshold is crossed, the state transitions from **CLOSED** or **DEGRADED** to **OPEN**, immediately short-circuiting further requests to that provider until the `resetTimeout` expires.

### What is the difference between connection cooldown and model lockout?

Connection cooldown operates at the failure-type level within the circuit breaker, applying specific back-off durations based on error classification (such as 429 rate limits versus 500 errors). Model lockout operates at the individual model-connection level through [`src/domain/lockoutPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/lockoutPolicy.ts), completely excluding specific model instances from routing after repeated failures. Cooldown delays requests to a provider, while lockout removes specific models from the candidate pool.

### Can circuit breaker state persist across application restarts?

Yes, the circuit breaker implementation in OmniRoute supports persistence through [`src/lib/db/domainState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/domainState.ts), which stores breaker state in SQLite. Similarly, model lockouts persist via [`src/lib/db/lockoutState.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/lockoutState.ts). This ensures that resilience decisions survive application restarts, preventing immediate re-quota-exhausted providers or re-locked models from receiving traffic before the system stabilizes.

### How does the half-open state prevent thundering herd problems?

The **HALF_OPEN** state limits probe traffic through the `halfOpenRequests` parameter (typically set to 1). When the `resetTimeout` expires, the breaker allows only this specific number of test requests to pass through. Successful probes restore normal operation (**CLOSED** state), while failed probes immediately reopen the circuit. This prevents all waiting requests from simultaneously hitting a recovering provider.