How OmniRoute Model Lockouts Enable Per-Model Quota Management

Model lockouts provide per-model, per-connection isolation in OmniRoute, ensuring a single exhausted model cannot disable an entire provider connection while preserving availability for other models.

OmniRoute implements model lockouts as a fine-grained quota management mechanism that isolates failures to individual models rather than entire provider connections. This article examines how the open-source routing engine detects, records, and utilizes lockout states to maintain resilient AI inference pipelines.

What Are Model Lockouts in OmniRoute?

A model lockout is a lightweight in-memory state that attaches to a specific provider connection and model identifier. When upstream quota limits trigger specific HTTP error codes, OmniRoute records a temporary lockout rather than marking the whole connection unavailable.

The system distinguishes between two failure modes:

• Per-model lockout: Isolates a single model on a connection
• Connection cooldown: Disables the entire connection across all models

This distinction preserves routing flexibility when only one model exhausts its quota while others remain operational.

Triggering Model Lockouts in the Auth Pipeline

The lockout detection logic resides in src/sse/services/auth.ts (lines 1247-1265). The request pipeline evaluates each response to determine whether a model-specific quota problem occurred.

Error Conditions That Trigger Lockouts

OmniRoute applies per-model lockouts when responses return HTTP 403, 404, or 429 status codes that indicate model-specific quota exhaustion rather than global connection failures.

// src/sse/services/auth.ts – lines 1247-1265
if (
  requestedModel &&
  (isModelLocked(provider, c.id, requestedModel) ||
    isAlibabaModelFreeDrained(
      provider,
      c.providerSpecificData as Record<string, unknown>,
      requestedModel
    ))
) {
  connectionFilterStatus.set(c.id, "modelLocked");
  if (provider === "antigravity" && getQuotaScopeLabelForProvider(provider, requestedModel) === "family") {
    familyLockedCount += 1;
  } else {
    modelLockedCount += 1;
  }
  return false;   // skip this connection for the current model
}

The isModelLocked() check consults the current lockout state, while isAlibabaModelFreeDrained() handles provider-specific quota detection for Alibaba's free tier models.

Recording and Storing Lockout State

When a lockout triggers, the markAccountUnavailable() helper constructs a ModelLockoutInfo object with three key properties:

• reason: The classification string (e.g., "quota_exhausted")
• remainingMs: Milliseconds until the lockout expires
• errorCode: The specific HTTP status that triggered the lockout

The storage structure uses a nested map keyed by provider → connectionId → model, enabling O(1) lookups during routing decisions.

Maximum Cooldown Enforcement

Lockout durations respect the maxCooldownMs setting defined in src/lib/db/settings/lkgp.ts. This ceiling prevents aggressive upstream rate limits from creating indefinite lockouts, ensuring the system recovers automatically.

// Example: Manually forcing a lockout (used in tests)
import { recordModelLockout } from "@/sse/services/auth";

await recordModelLockout({
  provider: "openai",
  connectionId: "conn-1234",
  model: "gpt-4o-mini",
  reason: "quota_exhausted",
  retryAfterMs: 60_000,   // lock for 1 minute
});

Routing Decisions with Active Lockouts

The routing layer—spanning combo selectors, single-model handlers, and the health autopilot—consults getModelLockoutInfo() before selecting connections. An active lockout filters the connection only for the locked model, leaving it eligible for other models.

// Example: Checking if a model is locked before sending a request
import { getModelLockoutInfo } from "@/sse/services/auth";

const lockout = getModelLockoutInfo("openai", "conn-1234", "gpt-4o-mini");
if (lockout && lockout.remainingMs > 0) {
  // Skip this connection or choose a fallback model
}

Debug logs expose lockout evaluations for observability:

// src/sse/services/auth.ts – lines 1249-1265 (excerpt)
const lockout = getModelLockoutInfo(provider, c.id, requestedModel);
log.debug(
  "AUTH",
  allowSuppressedConnections
    ? `  → ${c.id?.slice(0, 8)} | retained model lockout for ${requestedModel} (${lockout?.remainingMs || 0}ms remaining) for combo live test`
    : `  → ${c.id?.slice(0, 8)} | model-locked for ${requestedModel} (${lockout?.remainingMs || 0}ms remaining)`
);

Retry-After Header Support and Expiration

The lockout system integrates with upstream Retry-After headers through src/sse/services/cooldownAwareRetry.ts. When providers include this header, OmniRoute parses the value and sets remainingMs accordingly, enabling precise retry scheduling aligned with quota reset windows.

Automatic expiration occurs on the next connection evaluation after remainingMs reaches zero. No background cleanup process is required—the lazy evaluation design minimizes memory overhead for idle lockouts.

Operational Visibility and Remediation

Runtime Dashboard

The operator UI surfaces active model lockouts under the "Model lockouts" section (i18n key emptyLockouts). Operators can inspect:

• Which models are locked per connection
• Remaining lockout duration
• The triggering error reason

Health Autopilot Integration

src/lib/monitoring/providerHealthAutopilot.ts exposes lockouts as actionable items with a "Clear model lockout" button. This executes the clearModelLockout action to delete the in-memory entry immediately, useful for manual recovery or testing scenarios.

Summary

• Model lockouts isolate quota failures per model, preventing single-model exhaustion from disabling entire provider connections
• Detection occurs in src/sse/services/auth.ts via HTTP status code analysis and provider-specific heuristics
• Storage uses a nested map structure (provider → connectionId → model) with ModelLockoutInfo metadata
• Routing respects lockouts through getModelLockoutInfo() while keeping connections available for unlocked models
• Automatic expiration and Retry-After header support enable self-healing without operator intervention
• Operational tools in the health autopilot provide visibility and manual override capabilities

Frequently Asked Questions

What HTTP status codes trigger a model lockout versus a full connection cooldown?

OmniRoute triggers per-model lockouts on HTTP 403, 404, and 429 responses that indicate model-specific quota exhaustion. Full connection cooldowns apply to authentication failures, network errors, or provider-wide outages. The distinction is determined in src/sse/services/auth.ts (lines 1247-1265) based on response metadata and provider-specific detection logic like isAlibabaModelFreeDrained().

How long do model lockouts last?

Lockout duration defaults to the value specified in retryAfterMs or derived from upstream Retry-After headers, capped at maxCooldownMs from src/lib/db/settings/lkgp.ts. This ceiling prevents indefinite lockouts from misbehaving providers. The exact duration is stored in ModelLockoutInfo.remainingMs and decremented during routing evaluations.

Can operators manually clear a model lockout?

Yes. The health autopilot in src/lib/monitoring/providerHealthAutopilot.ts exposes a "Clear model lockout" button that invokes clearModelLockout to delete the in-memory entry immediately. This is useful for testing, emergency recovery, or correcting false-positive lockouts without waiting for expiration.

Do model lockouts affect other models on the same connection?

No. Per-model lockouts isolate failures to specific model identifiers. A connection with gpt-4o-mini locked remains eligible for routing requests to gpt-4o, gpt-3.5-turbo, or other models. The connectionFilterStatus.set(c.id, "modelLocked") call in auth.ts (line 1250) marks the filter reason without disabling the connection globally.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}