# Understanding OmniRoute's Three Resilience Layers: Provider Breaker, Connection Cooldown, and Model Lockout

> Discover OmniRoute's three resilience layers: provider breaker, connection cooldown, and model lockout. Isolate failures and ensure routing performance without service disruption.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-19

---

**OmniRoute implements three specialized resilience layers—provider circuit breaker, connection cooldown, and model lockout—to isolate failures and maintain routing performance without disabling entire services.**

The **OmniRoute** open-source routing proxy (by diegosouzapw/OmniRoute) protects LLM request pipelines through a tiered defense system. Each resilience layer targets a specific failure scope, enabling granular control over how the system responds to provider outages, rate limits, and model-specific errors.

## The Three Resilience Layers Explained

OmniRoute's resilience architecture operates on three distinct scopes:

| Layer | Scope | Primary Failure Mode |
|-------|-------|-------------------|
| **Provider Circuit Breaker** | Entire provider (e.g., `openai`, `anthropic`) | Upstream service degradation |
| **Connection Cooldown** | Single connection/account/key | Rate limits, auth errors |
| **Model Lockout** | Provider + connection + model triple | Per-model quotas, missing models |

This separation prevents over-reaction—a temporary model quota outage won't trigger a full provider shutdown.

## Layer 1: Provider Circuit Breaker

The **provider circuit breaker** stops all traffic to a provider when systemic failures exceed thresholds, protecting downstream latency from cascading degradation.

### Four-State State Machine

The breaker cycles through states defined in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts):

- **`CLOSED`** — Normal operation, requests pass through
- **`DEGRADED`** — Elevated errors, reduced but non-zero traffic allowed
- **`OPEN`** — Hard stop; all requests fast-fail for configured timeout
- **`HALF_OPEN`** — Probe request tests recovery after timeout expires

State transitions are lazy: the breaker only evaluates `HALF_OPEN` on the next status check after the `OPEN` timeout expires.

### Core Implementation Files

- **CircuitBreaker class**: [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)
- **Request pipeline integration**: [`src/sse/handlers/chatHelpers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chatHelpers.ts), [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts)
- **Monitoring endpoint**: `GET /api/monitoring/health` in [`src/app/api/monitoring/health/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/monitoring/health/route.ts)
- **Manual reset API**: `POST /api/resilience/reset` in [`src/app/api/resilience/reset/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/reset/route.ts)

```typescript
// Inspect provider breaker status programmatically
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';

const breaker = getCircuitBreaker('openai');
console.log(breaker.getStatus());
// => { name: 'openai', state: 'CLOSED', failureCount: 0, ... }

```

```typescript
// Administrative reset of all breakers
import { resetAllCircuitBreakers } from '@/shared/utils/circuitBreaker';

resetAllCircuitBreakers(); // Clears breaker state and database rows

```

## Layer 2: Connection Cooldown

When a **specific API key** hits rate limits or authentication errors, **connection cooldown** temporarily suspends that key while preserving other keys for the same provider.

### How Connection Cooldown Works

The mechanism isolates failures to individual credentials without provider-wide impact. Key functions in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) handle marking and selection:

- `markAccountUnavailable()` — Places a connection into cooldown with configurable duration
- `getProviderCredentials*()` — Skips cooling-down connections during credential selection

The `checkFallbackError()` function in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) evaluates whether errors trigger cooldown eligibility.

### Configuration and Defaults

Cooldown behavior is configurable in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts):

- Base cooldown durations
- Exponential backoff multipliers
- Maximum backoff caps

```typescript
// Manually trigger cooldown after rate limit detection
import { markAccountUnavailable, getProviderCredentials } from '@/sse/services/auth';

const credential = getProviderCredentials('openai', 'key-abc123');
if (credential) {
  markAccountUnavailable(credential, {
    rateLimitedUntil: Date.now() + 15_000, // 15 second cooldown
    testStatus: 'unavailable',
    backoffLevel: 1,
  });
}

```

## Layer 3: Model Lockout

The most granular layer, **model lockout**, disables specific model + connection combinations. This prevents over-penalization when only one model in a multi-model connection experiences quota exhaustion or availability issues.

### When Model Lockout Activates

Model lockout applies to scenarios including:

- Per-model API quotas (common in OpenAI tiered plans)
- Locally-hosted models that become unavailable
- Mode-specific restrictions (e.g., disabling vision models temporarily)

### Implementation and APIs

Core functions in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts):

- `lockModel()` — Creates a timed lockout entry
- `clearModelLock()` — Removes lockout manually or on expiration

Dashboard components provide operational visibility:

- [`ModelCooldownsCard.tsx`](https://github.com/diegosouzapw/OmniRoute/blob/main/ModelCooldownsCard.tsx) — Read-only active lockout list
- [`ModelLockoutCard.tsx`](https://github.com/diegosouzapw/OmniRoute/blob/main/ModelLockoutCard.tsx) — Configuration and manual re-enable interface

REST endpoints at `src/app/api/resilience/model-cooldowns/*`:

- `GET /api/resilience/model-cooldowns` — List all active lockouts
- `DELETE /api/resilience/model-cooldowns` — Clear specific or all lockouts

```typescript
// Lock a specific model after quota exhaustion
import { lockModel } from '@/open-sse/services/accountFallback';

await lockModel({
  provider: 'anthropic',
  connectionId: 'conn-def456',
  model: 'claude-2.0',
  reason: 'quota_exhausted',
  expiresAt: Date.now() + 120_000, // 2 minute lockout
});

```

## Debugging Resilience Issues

When investigating routing failures, examine layers in order of scope:

1. **Check provider breaker state** — Query `/api/monitoring/health` or call `getCircuitBreaker(provider).getStatus()`
2. **Inspect connection cooldowns** — Review credential availability in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) logs
3. **Verify model lockouts** — Query `GET /api/resilience/model-cooldowns` for targeted restrictions

Each layer generates distinct telemetry, enabling precise root cause identification without disabling healthy infrastructure.

## Summary

- **Provider circuit breaker** ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)) protects against systemic provider failures through a four-state machine with lazy recovery transitions
- **Connection cooldown** ([`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts)) isolates individual API keys experiencing rate limits while maintaining provider availability
- **Model lockout** ([`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts)) enables per-model granularity for quota and availability management

These three resilience layers form OmniRoute's defense-in-depth strategy, ensuring graceful degradation without unnecessary service disruption.

## Frequently Asked Questions

### How do I manually reset a tripped provider circuit breaker?

Call `resetAllCircuitBreakers()` from [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) or POST to `/api/resilience/reset`. This clears all breaker states and database persistence. Individual breaker reset is not exposed—design intent requires addressing the underlying provider issue before restoration.

### What causes a connection to enter cooldown versus triggering the provider breaker?

Connection cooldown activates on **credential-specific errors** (401/429 on a single key) via `markAccountUnavailable()`. The provider breaker trips only when **aggregated failures across all keys** for that provider exceed thresholds, indicating upstream service degradation rather than isolated key issues.

### Can model lockouts expire automatically or require manual clearing?

Both mechanisms exist. The `lockModel()` function accepts an `expiresAt` timestamp for automatic expiration. Alternatively, use the `DELETE /api/resilience/model-cooldowns` endpoint or dashboard [`ModelLockoutCard.tsx`](https://github.com/diegosouzapw/OmniRoute/blob/main/ModelLockoutCard.tsx) for immediate manual clearance before timeout.