# How OmniRoute's Circuit Breaker, Connection Cooldown, and Model Lockout Interact

> Learn how OmniRoute's circuit breaker, connection cooldown, and model lockout work together to enhance request routing resilience. Discover provider protection strategies.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-08-21

---

**OmniRoute protects request routing through three complementary resilience layers: provider-level circuit breakers that exclude failing providers entirely, connection cooldowns that temporarily disable specific account keys, and model lockouts that isolate individual models while keeping connections active for other workloads.**

OmniRoute is an open-source AI request routing system that implements sophisticated fault tolerance to handle transient failures and rate limits across multiple LLM providers. According to the diegosouzapw/OmniRoute source code, the platform uses a hierarchical protection system where **circuit breakers**, **connection cooldowns**, and **model lockouts** operate at different granularities to ensure high availability without over-penalizing healthy resources.

## The Three Resilience Layers

OmniRoute implements defense in depth through three distinct mechanisms defined in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) and [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts).

### Provider Circuit Breaker

The **provider circuit breaker** operates at the broadest scope, monitoring all accounts under a single provider (such as OpenAI or Gemini). Implemented in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), this breaker trips when a provider repeatedly returns hard-failure status codes (`408`, `429`, `500`, `502`, `503`, `504`). While the breaker state is **OPEN**, the provider is completely excluded from combo routing, regardless of individual connection health.

### Connection Cooldown

The **connection cooldown** targets a single account or API key. After transient errors such as generic `429` rate limits or `500` server errors, the connection is marked unavailable for a calculated backoff period stored in `rateLimitedUntil`. The functions `setConnectionRateLimitUntil` and `markAccountUnavailable` in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) handle this logic, ensuring all models on that specific connection are skipped until the cooldown expires.

### Model Lockout

The **model lockout** provides the finest granularity, targeting a specific provider + connection + model combination. For providers enforcing per-model quotas (such as Gemini, NVIDIA, or Ollama-cloud), a failing model is locked out via `lockModel` while the underlying connection remains usable for other models. The `isModelLocked` and `getModelLockoutInfo` functions in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) manage these restrictions.

## How the Layers Interact in the Request Pipeline

Understanding the precedence and interaction rules is critical for debugging routing behavior in production environments.

### Provider Circuit Breaker Dominates

If a provider's circuit breaker is **OPEN**, both connection cooldowns and model lockouts are ignored because the provider is already excluded from routing. This check occurs early in the request pipeline within [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) and the combo candidate filter, ensuring no wasted requests are sent to unhealthy providers.

### Connection Cooldown vs. Model Lockout

When a request fails, `markAccountUnavailable` determines whether to apply a connection-wide cooldown or a per-model lockout based on the error classification:

- **Per-model errors** (such as `404` for a single model or `429` classified as *quota-exhausted*) trigger `lockModelIfPerModelQuota`. The connection itself stays healthy, allowing other models on the same key to be used.
- **Connection-wide errors** (such as generic `429` rate-limits or `500` errors) trigger `setConnectionRateLimitUntil`, skipping all models on that connection until the cooldown expires.

### Hybrid Error Classification

Some errors require additional inspection. For `429` responses containing a *quota-exhausted* body, the system calls `shouldMarkAccountExhaustedFrom429`. If the provider does not use per-model quotas (like standard OpenAI accounts), the error falls back to a connection cooldown; otherwise, it becomes a model lockout.

### Combo Routing Skip Logic

Before attempting a target, the combo routing code in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) performs the following checks:

```typescript
if (isModelLocked(provider, target.connectionId || "", rawModel)) {
    log.info("COMBO", `Skipping retry for ${modelStr} — model lockout active`);
    continue; // model is locked, connection stays usable
}
// later, after a failure:
if (fallbackResult.shouldCooldown) {
    // record a connection-wide cooldown
}

```

This logic ensures locked models are not retried while other models on the same connection remain candidates. The final skip decisions are implemented in [`open-sse/services/combo/targetResolution.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/targetResolution.ts).

### Automatic Recovery

Both cooldowns and lockouts use **lazy recovery**. The next time the connection or model is inspected, the stored `rateLimitedUntil` or `until` timestamp is compared to `Date.now()`. If the time has passed, the entry clears automatically via `clearAllModelLockouts` or `resetCooldownFailureCount`.

## Practical Code Examples

### Checking System State

Inspect the current status of all three protection layers using the following utilities:

```typescript
import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";
import { isModelLocked, getModelLockoutInfo } from "@omniroute/open-sse/services/accountFallback";

// Provider circuit-breaker status
const breaker = getCircuitBreaker("openai");
console.log(`OpenAI breaker state: ${breaker.state}`);

// Connection cooldown (rateLimitedUntil) – read directly from DB
const conn = await db.providers.getConnection("openai", connectionId);
console.log(`Connection cooldown until: ${conn.rateLimitedUntil}`);

// Model lockout
const locked = isModelLocked("openai", connectionId, "gpt-4o-mini");
console.log(`Model gpt-4o-mini locked? ${locked}`);

if (locked) {
  const info = getModelLockoutInfo("openai", connectionId, "gpt-4o-mini");
  console.log(`Lockout expires in ${info?.remainingMs} ms, reason: ${info?.reason}`);
}

```

### Recording Per-Model Failures

When a per-model error occurs on a quota-enforced provider, trigger a model-specific lockout:

```typescript
import { markAccountUnavailable } from "@omniroute/open-sse/services/auth";

// Simulate a 404 on a per-model-quota provider (e.g., Gemini)
await markAccountUnavailable(
  connectionId,
  404,
  "model not found",
  "gemini",
  "gemini-1.5-flash"
);
// Creates a model lockout (~30s by default) but leaves the
// connection usable for other Gemini models.

```

### Triggering Connection-Wide Cooldowns

For errors affecting the entire connection, omit the model parameter to trigger a full cooldown:

```typescript
await markAccountUnavailable(
  connectionId,
  429,
  "Too many requests",
  "openai"
  // No model argument → connection-wide cooldown
);

```

This sets `rateLimitedUntil` on the connection according to the provider profile (e.g., 60 seconds for OAuth, 30 seconds for API-key accounts).

### Resetting Lockouts

Clear all active model lockouts programmatically without affecting connection cooldowns:

```typescript
import { clearAllModelLockouts } from "@omniroute/open-sse/services/accountFallback";

// Clears every in-memory model lockout; connection cooldowns remain untouched.
clearAllModelLockouts();

```

## Summary

- **Provider circuit breakers** act as the first line of defense, excluding entire providers when systemic failures occur ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)).
- **Connection cooldowns** isolate specific API keys after transient errors, managed by `setConnectionRateLimitUntil` in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts).
- **Model lockouts** provide granular protection for per-model quotas via `lockModel`, allowing connections to serve other models while one model recovers.
- **Interaction hierarchy** places circuit breakers above cooldowns and lockouts; if a breaker is open, lower-level restrictions are irrelevant.
- **Lazy recovery** ensures system efficiency by clearing expired restrictions only when they are next accessed, avoiding background cleanup processes.

## Frequently Asked Questions

### What happens when a provider's circuit breaker is open but only one model is failing?

When a provider circuit breaker enters the **OPEN** state, the entire provider is excluded from routing regardless of individual model health. According to the implementation in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts), this check occurs before evaluating individual connections or models, meaning both connection cooldowns and model lockouts are effectively ignored until the breaker resets.

### How does OmniRoute distinguish between a connection-wide cooldown and a per-model lockout?

The system distinguishes these based on error type and provider capabilities. In [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts), the `markAccountUnavailable` function checks if an error qualifies as per-model (such as `404` or quota-exhausted `429` responses) and whether the provider supports per-model quotas. If both conditions are true, it calls `lockModelIfPerModelQuota`; otherwise, it invokes `setConnectionRateLimitUntil` for the entire connection.

### How long do connection cooldowns and model lockouts last?

Connection cooldowns use provider-specific durations defined in configuration profiles (typically 30 seconds for API-key accounts and 60 seconds for OAuth). Model lockouts default to approximately 30 seconds but vary based on the error classification. Both use timestamp-based expiration checked lazily against `Date.now()` when the resource is next accessed.

### Can I manually reset a model lockout without restarting the service?

Yes. Import `clearAllModelLockouts` from `@omniroute/open-sse/services/accountFallback` to immediately clear all in-memory model lockouts. This function preserves connection cooldowns and circuit breaker states, allowing surgical recovery of specific models without affecting the broader system's failure tracking.