# How OmniRoute Model Lockouts Enable Per-Model Quota Management

> Discover how OmniRoute model lockouts achieve per-model quota management. Ensure provider connections remain available even if one model is exhausted, enhancing overall system resilience.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-13

---

**Model lockouts provide per-model, per-connection isolation in OmniRoute, ensuring a single exhausted model cannot disable an entire provider connection while preserving availability for other models.**

OmniRoute implements **model lockouts** as a fine-grained quota management mechanism that isolates failures to individual models rather than entire provider connections. This article examines how the open-source routing engine detects, records, and utilizes lockout states to maintain resilient AI inference pipelines.

## What Are Model Lockouts in OmniRoute?

A **model lockout** is a lightweight in-memory state that attaches to a specific provider connection and model identifier. When upstream quota limits trigger specific HTTP error codes, OmniRoute records a temporary lockout rather than marking the whole connection unavailable.

The system distinguishes between two failure modes:
- **Per-model lockout**: Isolates a single model on a connection
- **Connection cooldown**: Disables the entire connection across all models

This distinction preserves routing flexibility when only one model exhausts its quota while others remain operational.

## Triggering Model Lockouts in the Auth Pipeline

The lockout detection logic resides in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) (lines 1247-1265). The request pipeline evaluates each response to determine whether a model-specific quota problem occurred.

### Error Conditions That Trigger Lockouts

OmniRoute applies per-model lockouts when responses return HTTP **403**, **404**, or **429** status codes that indicate model-specific quota exhaustion rather than global connection failures.

```ts
// src/sse/services/auth.ts – lines 1247-1265
if (
  requestedModel &&
  (isModelLocked(provider, c.id, requestedModel) ||
    isAlibabaModelFreeDrained(
      provider,
      c.providerSpecificData as Record<string, unknown>,
      requestedModel
    ))
) {
  connectionFilterStatus.set(c.id, "modelLocked");
  if (provider === "antigravity" && getQuotaScopeLabelForProvider(provider, requestedModel) === "family") {
    familyLockedCount += 1;
  } else {
    modelLockedCount += 1;
  }
  return false;   // skip this connection for the current model
}

```

The `isModelLocked()` check consults the current lockout state, while `isAlibabaModelFreeDrained()` handles provider-specific quota detection for Alibaba's free tier models.

## Recording and Storing Lockout State

When a lockout triggers, the `markAccountUnavailable()` helper constructs a `ModelLockoutInfo` object with three key properties:

- **reason**: The classification string (e.g., `"quota_exhausted"`)
- **remainingMs**: Milliseconds until the lockout expires
- **errorCode**: The specific HTTP status that triggered the lockout

The storage structure uses a nested map keyed by `provider → connectionId → model`, enabling O(1) lookups during routing decisions.

### Maximum Cooldown Enforcement

Lockout durations respect the `maxCooldownMs` setting defined in [`src/lib/db/settings/lkgp.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/settings/lkgp.ts). This ceiling prevents aggressive upstream rate limits from creating indefinite lockouts, ensuring the system recovers automatically.

```ts
// Example: Manually forcing a lockout (used in tests)
import { recordModelLockout } from "@/sse/services/auth";

await recordModelLockout({
  provider: "openai",
  connectionId: "conn-1234",
  model: "gpt-4o-mini",
  reason: "quota_exhausted",
  retryAfterMs: 60_000,   // lock for 1 minute
});

```

## Routing Decisions with Active Lockouts

The routing layer—spanning combo selectors, single-model handlers, and the health autopilot—consults `getModelLockoutInfo()` before selecting connections. An active lockout filters the connection **only for the locked model**, leaving it eligible for other models.

```ts
// Example: Checking if a model is locked before sending a request
import { getModelLockoutInfo } from "@/sse/services/auth";

const lockout = getModelLockoutInfo("openai", "conn-1234", "gpt-4o-mini");
if (lockout && lockout.remainingMs > 0) {
  // Skip this connection or choose a fallback model
}

```

Debug logs expose lockout evaluations for observability:

```ts
// src/sse/services/auth.ts – lines 1249-1265 (excerpt)
const lockout = getModelLockoutInfo(provider, c.id, requestedModel);
log.debug(
  "AUTH",
  allowSuppressedConnections
    ? `  → ${c.id?.slice(0, 8)} | retained model lockout for ${requestedModel} (${lockout?.remainingMs || 0}ms remaining) for combo live test`
    : `  → ${c.id?.slice(0, 8)} | model-locked for ${requestedModel} (${lockout?.remainingMs || 0}ms remaining)`
);

```

## Retry-After Header Support and Expiration

The lockout system integrates with upstream `Retry-After` headers through [`src/sse/services/cooldownAwareRetry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/cooldownAwareRetry.ts). When providers include this header, OmniRoute parses the value and sets `remainingMs` accordingly, enabling precise retry scheduling aligned with quota reset windows.

Automatic expiration occurs on the next connection evaluation after `remainingMs` reaches zero. No background cleanup process is required—the lazy evaluation design minimizes memory overhead for idle lockouts.

## Operational Visibility and Remediation

### Runtime Dashboard

The operator UI surfaces active model lockouts under the "Model lockouts" section (i18n key `emptyLockouts`). Operators can inspect:
- Which models are locked per connection
- Remaining lockout duration
- The triggering error reason

### Health Autopilot Integration

[`src/lib/monitoring/providerHealthAutopilot.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthAutopilot.ts) exposes lockouts as actionable items with a **"Clear model lockout"** button. This executes the `clearModelLockout` action to delete the in-memory entry immediately, useful for manual recovery or testing scenarios.

## Summary

- **Model lockouts isolate quota failures per model**, preventing single-model exhaustion from disabling entire provider connections
- **Detection occurs in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts)** via HTTP status code analysis and provider-specific heuristics
- **Storage uses a nested map structure** (`provider → connectionId → model`) with `ModelLockoutInfo` metadata
- **Routing respects lockouts** through `getModelLockoutInfo()` while keeping connections available for unlocked models
- **Automatic expiration** and `Retry-After` header support enable self-healing without operator intervention
- **Operational tools** in the health autopilot provide visibility and manual override capabilities

## Frequently Asked Questions

### What HTTP status codes trigger a model lockout versus a full connection cooldown?

OmniRoute triggers per-model lockouts on HTTP **403**, **404**, and **429** responses that indicate model-specific quota exhaustion. Full connection cooldowns apply to authentication failures, network errors, or provider-wide outages. The distinction is determined in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) (lines 1247-1265) based on response metadata and provider-specific detection logic like `isAlibabaModelFreeDrained()`.

### How long do model lockouts last?

Lockout duration defaults to the value specified in `retryAfterMs` or derived from upstream `Retry-After` headers, capped at `maxCooldownMs` from [`src/lib/db/settings/lkgp.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/settings/lkgp.ts). This ceiling prevents indefinite lockouts from misbehaving providers. The exact duration is stored in `ModelLockoutInfo.remainingMs` and decremented during routing evaluations.

### Can operators manually clear a model lockout?

Yes. The health autopilot in [`src/lib/monitoring/providerHealthAutopilot.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthAutopilot.ts) exposes a **"Clear model lockout"** button that invokes `clearModelLockout` to delete the in-memory entry immediately. This is useful for testing, emergency recovery, or correcting false-positive lockouts without waiting for expiration.

### Do model lockouts affect other models on the same connection?

No. Per-model lockouts isolate failures to specific model identifiers. A connection with `gpt-4o-mini` locked remains eligible for routing requests to `gpt-4o`, `gpt-3.5-turbo`, or other models. The `connectionFilterStatus.set(c.id, "modelLocked")` call in [`auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/auth.ts) (line 1250) marks the filter reason without disabling the connection globally.