# OmniRoute Resilience Layers: Circuit Breaker, Connection Cooldown, and Model Lockout Explained

> Discover OmniRoute resilience layers like circuit breaker, connection cooldown, and model lockout. Learn how they prevent cascading outages and isolate upstream failures. Protect your system today.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-07-13

---

**OmniRoute implements a three-layer resilience system using provider-wide circuit breakers, per-failure-kind connection cooldowns, and model-level lockouts to isolate upstream failures and prevent cascading outages.**

OmniRoute is an open-source AI routing layer designed to maintain service continuity during upstream provider instability. The codebase in `diegosouzapw/OmniRoute` implements a hierarchical defense mechanism that isolates failures at the provider, connection, and model levels. These resilience layers ensure that transient errors or rate limits from one upstream source do not degrade the entire routing pipeline.

## The Three Resilience Layers

OmniRoute protects the routing pipeline through a granular, hierarchical system that operates at increasing levels of specificity.

### Provider-Wide Circuit Breaker

The first layer guards all requests targeting a specific provider using a state-machine-based circuit breaker implemented in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts). This component tracks consecutive failures across any model on that provider and implements a **CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED** state transition.

When failures exceed the configurable `failureThreshold` (default 5), the breaker transitions to **OPEN**, short-circuiting further calls to that provider. After the `resetTimeout` period expires, the state shifts to **HALF_OPEN**, allowing a limited number of probe requests (`halfOpenRequests`). Successful probes close the circuit, while failed probes reopen it.

The circuit breaker exposes `canExecute()` to check availability and `execute()` to wrap upstream calls with automatic failure tracking and state management.

### Connection Cooldown (Per-Failure-Kind Back-off)

The second layer adds per-failure-type granularity within the same [`circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/circuitBreaker.ts) file. Instead of applying a uniform cooldown, OmniRoute uses a `cooldownByKind` map and `classifyError` logic to assign specific back-off periods to different error categories such as *rate-limit*, *quota-exhausted*, or *transient* failures.

When a failure occurs, the breaker calculates an `_effectiveCooldown` by selecting the most restrictive timeout between the generic `resetTimeout` and the specific cooldown for that failure kind. This prevents "hammering" a provider with repeated requests when it returns HTTP 429 or quota-exhausted errors, allowing aggressive retries for transient network errors while enforcing longer delays for rate limits.

### Model-Level Lockout

The third layer operates at the individual model-connection pair level, isolating specific misbehaving endpoints without disabling the entire provider. Implemented in [`src/domain/lockoutPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/lockoutPolicy.ts) and consumed by the account-fallback service in [`src/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/services/accountFallback.ts), this layer tracks failures per identifier using a sliding window approach.

The system counts failures up to `maxAttempts` within an `attemptWindowMs` period. When the threshold is crossed, a lockout record is persisted to the `domain_lockout_state` table with a `lockoutDurationMs` expiration. While locked, the model is excluded from combo resolution and selection. A successful request via `recordSuccess()` or an explicit `forceUnlock` clears the lockout immediately.

## How the Layers Interact

These resilience mechanisms operate hierarchically to provide defense in depth:

1. **Circuit breaker** determines if a provider is usable at all. If the breaker is **OPEN**, the provider is excluded entirely from combo resolution.
2. When the circuit is **CLOSED** or **DEGRADED**, the **connection cooldown** layer may still delay the next request based on the specific failure kind (`cooldownByKind`), applying the calculated `_effectiveCooldown` before allowing traffic.
3. Even when a provider is healthy, the **model lockout** layer can temporarily drop a specific model-connection pair (e.g., *Gemini-1.5-flash* on account *conn-123*) that repeatedly returns 429 or 404 errors, consulting the `domain_lockout_state` table during the combo selection process.

This architecture ensures that systemic outages trigger provider-wide isolation, rate-limit spikes trigger connection-specific delays, and individual model failures trigger targeted lockouts without affecting healthy upstream capacity.

## Implementation Examples

### Using the Circuit Breaker for Upstream Calls

The following pattern demonstrates how to wrap provider calls with circuit breaker protection:

```typescript
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

// Create or retrieve a breaker for the "openai" provider
const breaker = getCircuitBreaker('openai', {
  failureThreshold: 5,
  resetTimeout: 30_000,      // 30s default cooldown
  halfOpenRequests: 1,
  // Optional per-kind overrides (e.g., 429 → 2 min)
  cooldownByKind: { rate_limit: 120_000 },
});

async function callOpenAI(payload: any) {
  // Check if the circuit allows execution
  if (!breaker.canExecute()) {
    throw new Error('OpenAI is currently unavailable (circuit open)');
  }

  // Wrap the actual HTTP call
  return breaker.execute(async () => {
    // ... fetch() implementation
  });
}

```

### Applying Model-Level Lockout

When a specific model returns a 429 error, trigger a lockout via the account fallback service:

```typescript
import { markAccountUnavailable } from '@/services/accountFallback';

// Mark a specific model-connection as unavailable
await markAccountUnavailable({
  provider: 'gemini',
  connectionId: 'conn-123',
  model: 'gemini-1.5-flash',
  reason: 'rate_limit',
  // Lockout persists for the provider's default duration
});

// The combo resolver will automatically skip this locked model
const combo = await resolveComboTargets(...);

```

### Monitoring Resilience State

Inspect current system health using exported status functions:

```typescript
import { getAllCircuitBreakerStatuses } from '@/shared/utils/circuitBreaker';
import { getAllModelLockouts } from '@/domain/lockoutPolicy';

console.log('Provider breakers:', getAllCircuitBreakerStatuses());
console.log('Model lockouts:', await getAllModelLockouts());

```

## Summary

- **Provider-wide circuit breakers** in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) isolate entire providers after 5 consecutive failures, using a state machine with configurable `resetTimeout` and `halfOpenRequests`.
- **Connection cooldowns** provide per-failure-type back-off via `cooldownByKind`, allowing different delays for rate limits versus transient errors.
- **Model-level lockouts** in [`src/domain/lockoutPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/lockoutPolicy.ts) track failures per model-connection pair using sliding windows (`maxAttempts` within `attemptWindowMs`) and persist blocks to `domain_lockout_state`.
- The **account-fallback service** ([`src/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/services/accountFallback.ts)) consumes lockout data to exclude degraded models from routing decisions.
- These layers operate hierarchically: circuit breakers exclude unhealthy providers, cooldowns delay specific error types, and lockouts isolate individual models.

## Frequently Asked Questions

### How does the circuit breaker state machine transition between states?

The circuit breaker starts in **CLOSED** (normal operation). After `failureThreshold` consecutive failures (default 5), it transitions to **OPEN**, rejecting all requests. After `resetTimeout` milliseconds, it enters **HALF_OPEN**, allowing `halfOpenRequests` probe calls. Successful probes transition back to **CLOSED**, while failed probes return to **OPEN**. An intermediate **DEGRADED** state may also indicate elevated error rates before full opening.

### What is the difference between connection cooldown and the circuit breaker?

The **circuit breaker** is binary—it either allows all traffic or blocks all traffic to a provider based on aggregate failure counts. **Connection cooldown** operates within an open circuit or during degraded states, applying specific delays based on the *type* of error encountered (e.g., 120 seconds for rate limits vs. 30 seconds for generic errors). This prevents hammering providers with 429 errors while allowing faster retries for transient network blips.

### How long does a model lockout persist?

Model lockouts persist for the duration specified by `lockoutDurationMs` in the lockout policy, which varies by provider configuration. The lockout is stored in the `domain_lockout_state` table and automatically expires after the duration elapses. Alternatively, a successful request to that model-connection pair via `recordSuccess()` or an explicit `forceUnlock` call will immediately clear the lockout.

### Can resilience thresholds be customized per provider?

Yes. When retrieving a circuit breaker via `getCircuitBreaker()`, you can specify custom `failureThreshold`, `resetTimeout`, `halfOpenRequests`, and a `cooldownByKind` map to override defaults for specific error types. Similarly, model lockout parameters including `maxAttempts`, `attemptWindowMs`, and `lockoutDurationMs` are configurable per provider in the domain policy configuration.