# Connection Cooldown vs Model Lockout in OmniRoute: Key Differences Explained

> Understand the difference between Connection Cooldown and Model Lockout in OmniRoute. Learn how Connection Cooldown disables entire connections while Model Lockout isolates specific models for better error handling.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-08

---

**Connection Cooldown temporarily disables an entire provider connection (account + API key) after transient HTTP errors, while Model Lockout isolates only a specific model on that connection, allowing other models to continue serving requests.**

OmniRoute implements a three-tier resilience strategy to protect AI provider integrations from cascading failures. While the Provider Circuit Breaker acts as a coarse-grained safety net for entire providers, **Connection Cooldown** and **Model Lockout** operate at finer granularity to handle connection-wide and model-specific issues respectively. Understanding the distinction between these mechanisms is essential for debugging routing decisions and optimizing retry behavior in production deployments.

## What Is Connection Cooldown?

**Connection Cooldown** guards a single connection—defined as a specific account and API key pair—to a provider. It triggers when the connection experiences retryable failures such as HTTP 408 (Request Timeout), 429 (Too Many Requests), or any 5xx server error.

When activated, OmniRoute sets a `rateLimitedUntil` timestamp on the connection. The routing layer in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) checks this timestamp via `parseFutureDateMs(connection.rateLimitedUntil)` and excludes the connection from selection while the cooldown is active. Other connections to the same provider remain fully operational.

The cooldown duration is calculated using exponential backoff (`baseCooldownMs * 2**failureCount`) and can optionally honor upstream `Retry-After` headers when `useUpstreamRetryHints` is enabled in the resilience profile.

## What Is Model Lockout?

**Model Lockout** operates at a more granular level, guarding an individual model (or model family) on a specific connection. It triggers on per-model quota exhaustion (e.g., 429 errors with reason `rate_limited`), missing model errors (404), or provider-specific model-level limits.

The mechanism uses a composite key format `"provider:connectionId:model"` stored in a lockout map within [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts). When `lockModel()` is invoked, it stores an expiry timestamp (`newUntil = Date.now() + cooldownMs`) with optional metadata. The routing layer checks `isModelLocked(provider, connectionId, requestedModel)` before selecting a connection, and locked models are tracked separately via `modelLockedCount` with detailed logging of remaining lockout time.

Model lockouts are cleaned up automatically every 15 seconds via a lazy timer to prevent memory leaks.

## Key Differences Between Connection Cooldown and Model Lockout

| Aspect | Connection Cooldown | Model Lockout |
|--------|---------------------|---------------|
| **Scope** | Entire connection (account + key) | Single model on a specific connection |
| **Trigger** | HTTP 408/429/5xx, connection-level errors | Per-model quota limits, 404 missing model |
| **Impact** | All models on the connection are unavailable | Only the specific model is unavailable; other models work |
| **Storage** | `rateLimitedUntil` timestamp on connection object | Entry in `"provider:connectionId:model"` map |
| **Backoff** | Exponential with configurable steps | Fixed duration based on error context |

## How Connection Cooldown Works Under the Hood

### Configuration and Detection

The resilience profile builder in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) (lines 70-85) reads connection-level fields from settings including `baseCooldownMs`, `useUpstreamRetryHints`, and `maxBackoffSteps`. When a request fails with a retryable status, the connection's `rateLimitedUntil` field is updated to `now + cooldownMs`.

### Cooldown Calculation

The cooldown calculation (lines 381-385 in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts)) supports exponential backoff and upstream hint processing:

```typescript
// Exponential backoff: baseCooldownMs * 2^failureCount
const cooldownMs = baseCooldownMs * Math.pow(2, failureCount);

// Respect upstream Retry-After if enabled
if (useUpstreamRetryHints && retryAfterHeader) {
  return Math.max(cooldownMs, parseRetryAfter(retryAfterHeader));
}

```

### Code Example

```typescript
// Example: Connection enters cooldown after 429 error
await fetchProvider(request); // fails with HTTP 429

// OmniRoute automatically sets:
connection.rateLimitedUntil = Date.now() + cooldownMs;

// Subsequent routing checks skip this connection until timestamp expires

```

## How Model Lockout Works Under the Hood

### The Lockout Map Structure

Model lockouts are managed in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) (lines 18-22) using a Map with composite keys:

```typescript
// Key format: "provider:connectionId:model"
const lockoutKey = `${provider}:${connectionId}:${model}`;

// Stored value includes expiry and metadata
lockoutMap.set(lockoutKey, {
  until: Date.now() + cooldownMs,
  reason: 'rate_limited'
});

```

### Routing Integration

The [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) file (lines 12-16 and 1278-1297) implements the lockout check:

```typescript
if (isModelLocked(provider, connectionId, requestedModel)) {
  modelLockedCount++;
  continue; // Skip this connection for this specific model
}

```

When all connections are blocked by model cooldowns for a requested model, the system reports `allBlockedByModelCooldown` (lines 19-26), setting `cooldownScope` to `"model"` in API responses.

### Code Example

```typescript
// Example: Locking a specific model on a connection
lockModel(
  "openai",
  "conn-1234",
  "gpt-4o-mini",
  "rate_limited",      // reason from RateLimitReason enum
  30_000               // 30-second lockout duration
);

// Later requests for "gpt-4o-mini" on conn-1234 will fail isModelLocked() check
// But requests for "gpt-4o" on the same connection proceed normally

```

## When to Use Each Mechanism

Use **Connection Cooldown** when encountering generic transient errors that affect the whole account, such as rate-limit headers applied to the API key or temporary provider outages. This protects the routing engine from repeatedly hammering a flaky connection without penalizing specific models.

Use **Model Lockout** when errors are isolated to a single model, such as quota exhaustion for `gpt-4o-mini` while `gpt-4o` remains available, or when a model is removed from the provider's catalog. This allows the connection to continue serving other models while isolating the problematic one.

## Summary

- **Connection Cooldown** operates at the connection level (account + API key), triggered by HTTP 408/429/5xx errors, and temporarily disables the entire connection using exponential backoff and optional `Retry-After` support.
- **Model Lockout** operates at the model level on a specific connection, triggered by per-model quota errors or 404s, using a composite key map to exclude only the affected model while preserving connection availability for other models.
- Both mechanisms are orthogonal to the Provider Circuit Breaker and cooperate to provide fine-grained resilience, with `cooldownScope` fields indicating whether a routing failure stems from connection or model-level blocks.

## Frequently Asked Questions

### Can a connection be in cooldown while a model is locked on another connection?

Yes. These mechanisms are independent. Connection A can be in cooldown (affecting all its models) while Connection B has Model X locked (but still serves Model Y). Additionally, Model X could be locked on Connection B even while Connection A (which is not in cooldown) continues to serve Model X normally.

### How long do lockouts typically last?

Connection Cooldown durations are configured via `baseCooldownMs` in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) and grow exponentially with repeated failures (up to `maxBackoffSteps`). Model Lockout durations are typically shorter and context-specific—often 30-60 seconds for rate limits—though the exact value depends on the error response and hardcoded defaults in the resilience profile.

### Does Model Lockout affect all connections or just one?

Model Lockout affects only the specific connection where the error occurred. The lockout key includes `connectionId`, meaning `gpt-4o-mini` might be locked on Connection A but fully available on Connection B. This isolation prevents a single model's quota exhaustion from degrading service for that model across your entire OmniRoute deployment.

### Where are these resilience settings configured?

Default settings for both mechanisms reside in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts), including `baseCooldownMs`, `useUpstreamRetryHints`, and cleanup intervals. Runtime behavior is further controlled by the resilience profile builder in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts). For human-readable documentation, refer to [`docs/architecture/RESILIENCE_GUIDE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/architecture/RESILIENCE_GUIDE.md) sections "2. Connection Cooldown" and "3. Model Lockout".