# How OmniRoute's Key Rotation and Failover Mechanism Works Across Provider Accounts

> Discover how OmniRoute's key rotation and failover protects request throughput using per-account rotation, exponential backoff, and circuit breakers. Ensure reliability across provider accounts.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-16

---

**OmniRoute protects request throughput through three complementary layers: per-account rotation with configurable thresholds, connection-level exponential backoff, and provider-wide circuit breakers that automatically isolate failing credentials.**

OmniRoute is an open-source AI gateway that intelligently routes requests across multiple LLM providers. Its **key rotation and failover mechanism** ensures high availability by automatically retiring compromised accounts, cooling down rate-limited keys, and blocking entire providers when systemic failures occur. This article explains how these systems work based on the source code in `diegosouzapw/OmniRoute`.

## The Three-Layer Resilience Architecture

OmniRoute implements defense in depth through three coordinated mechanisms:

| Mechanism | Purpose | Core Implementation |
|-----------|---------|---------------------|
| **Per-account rotation (fallback)** | Decides when to retire an account and try a fresh credential | [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) — functions `gateFor`, `recordErrorAndCheckThreshold`, `evaluateRotationGate` |
| **Connection-level cooldown** | Applies exponential backoff to the specific key that failed | [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) — `recordProviderFailure`, `calculateBackoffCooldown` |
| **Provider-wide circuit breaker** | Blocks all keys for a provider when aggregated failures exceed thresholds | [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) with [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) |

## Runtime Rotation Configuration

All rotation behavior is controlled through [`rotationConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rotationConfig.ts), which reads environment variables and supports per-connection overrides.

```ts
// open-sse/services/rotationConfig.ts
export function getGlobalRotationConfig(): RotationConfig { … }

```

### Environment Variable Controls

| Variable | Default | Behavior |
|----------|---------|----------|
| `OMNIROUTE_ROTATE_ON_429` | `true` | Enable rotation on rate-limit (429) errors |
| `OMNIROUTE_ROTATE_429_THRESHOLD` | `1` | Errors required before rotation triggers |
| `OMNIROUTE_ROTATE_429_WINDOW_SECONDS` | `120` | Sliding window for error counting |

Per-connection overrides via `providerSpecificData.rotationOverrides` allow fine-tuning without restarts. See `resolveRotationConfig` (lines 60-88) for the merge logic.

## Decision Flow for Failed Requests

When a request fails, [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) executes this sequence:

```ts
// open-sse/services/accountFallback.ts
export function gateFor(status: number, account?: unknown) {
  const { rotationOverrides, rotationKey } = extractRotationContext(account);
  return evaluateRotationGate(
    status,
    resolveRotationConfig(rotationOverrides),
    rotationKey,
  );
}

```

The `evaluateRotationGate` function applies three checks in order:

1. **Master switch validation** — `isFallbackBlockedForStatus` returns `true` if rotation is globally disabled for this error class
2. **Threshold sliding window** — `recordErrorAndCheckThreshold` counts errors per `(key, status)` pair; rotation triggers when `threshold` errors occur within `windowMs`
3. **400-force fallback** — If `rotateOn400` is enabled, bad requests can trigger rotation using `rateLimitCooldownOverrideMs`

If all checks pass without triggering rotation, OmniRoute falls back to built-in heuristics like parsing `Retry-After` headers.

## Cooling Down Individual Keys

Rotated credentials enter a cooldown period computed by `calculateBackoffCooldown` in [`config/errorConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/config/errorConfig.ts):

```

backoffMs = min(baseCooldownMs * 2^(failureCount - 1), maxBackoffMs)

```

The cooldown state is dual-tracked:
- **In-memory**: Via `CircuitBreaker` instance per provider
- **Persistent**: In `providers.rateLimitedUntil` database column

```ts
// open-sse/services/accountFallback.ts
export function recordProviderFailure(
  provider: string | null | undefined,
  log?,
  connectionId?,
  profile?,
  opts?,
) {
  const breaker = configureProviderBreaker(provider, profile);
  if (!breaker?.canExecute()) return;
  breaker._onFailure();  // Updates circuit-breaker state
}

```

## Provider-Wide Circuit Breaker

Separate from per-key cooldowns, the **circuit breaker** aggregates failures across all keys for a provider. It deduplicates errors using:

```ts
// Constants in accountFallback.ts (lines 28-42)
const CONNECTION_FAILURE_DEDUP_MS = 5000;    // 5 seconds
const NETWORK_ERROR_DEDUP_MS = 10000;        // 10 seconds

```

Only distinct failures contribute to the threshold. When `failureThreshold` is exceeded, the breaker opens and triggers `providerCooldownMs`:

```ts
export function isProviderInCooldown(provider: string | null | undefined): boolean {
  const breaker = getProviderBreaker(provider);
  return breaker ? !breaker.canExecute() : false;
}

```

While **OPEN**, `recordProviderSuccess` does **not** reset the cooldown — the provider remains blocked until `resetTimeoutMs` expires.

## Per-Model Lockouts for Multiplexed Providers

Providers like Gemini and Codex multiplex multiple models on single credentials. OmniRoute can lock **individual models** without disabling the entire connection:

- `lockModelIfPerModelQuota` — applies model-specific lockout
- `isModelLocked` — checks lockout status before routing
- `recordModelLockoutFailure` — triggers exponential backoff per model

The `hasPerModelQuota` function (lines 145-162) determines whether a provider uses this behavior. Lockout entries are evicted by [`lockoutEviction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/lockoutEviction.ts) to prevent unbounded memory growth.

## Complete Rotation and Failover Pipeline

1. **Request arrives** → API route dispatches to [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts)
2. **Error classification** → `classifyErrorText` and `RateLimitReason` categorize the failure
3. **Rotation gate** → `gateFor` decides: rotate now, hold, or continue
4. **If rotation required**:
   - Mark credential cooldown via `recordProviderFailure`
   - Update provider circuit breaker if failure contributes to threshold
   - Optionally lock specific model via `lockModelIfPerModelQuota`
5. **Router selection** → [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) skips cooled credentials and selects next viable key

## Practical Code Examples

### Force Rotation on 429 for a Specific Account

```ts
import { gateFor } from '@/open-sse/services/accountFallback';
import { getGlobalRotationConfig } from '@/open-sse/services/rotationConfig';

const account = {
  id: 'account-123',
  providerSpecificData: {
    rotationOverrides: {
      rotateOn429: true,
      error429Threshold: 2,
      error429WindowSeconds: 60,
    },
  },
};

const decision = gateFor(429, account);
if (decision?.shouldFallback) {
  console.log('Rotate this key, cooldown:', decision.cooldownMs);
}

```

### Check Provider Status Before Dispatching

```ts
import { isProviderInCooldown, getProviderCooldownRemainingMs } from '@/open-sse/services/accountFallback';

const provider = 'openai';
if (isProviderInCooldown(provider)) {
  const remaining = getProviderCooldownRemainingMs(provider);
  console.warn(`Provider ${provider} is throttled for ${remaining} ms`);
} else {
  // Safe to send request
}

```

## Key Source Files

| File | Purpose |
|------|---------|
| [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) | Core fallback engine, rotation gate, circuit-breaker integration, model lockouts |
| [`open-sse/services/rotationConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/rotationConfig.ts) | Runtime configuration from env vars and per-connection overrides |
| [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) | Shared circuit-breaker implementation |
| [`open-sse/services/accountFallback/lockoutEviction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback/lockoutEviction.ts) | Stale model-lockout cleanup |
| [`open-sse/services/accountFallback/exactModelLock.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback/exactModelLock.ts) | Model-specific lockout key utilities |
| [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) | Default resilience parameters |

## Summary

- **Per-account rotation** uses configurable thresholds and sliding windows to retire failing credentials, implemented in [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) with rules from [`rotationConfig.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rotationConfig.ts)

- **Connection-level cooldown** applies exponential backoff to individual keys, storing state in both memory (`CircuitBreaker`) and database (`providers.rateLimitedUntil`)

- **Provider-wide circuit breaker** blocks all keys when aggregated failures exceed thresholds, with deduplication preventing false triggers

- **Per-model lockouts** enable granular control for multiplexed providers without disabling entire connections

- **Runtime configurability** via environment variables and `rotationOverrides` allows operators to tune behavior without code changes

## Frequently Asked Questions

### What triggers key rotation in OmniRoute?

Key rotation triggers when error counts exceed configured thresholds within a sliding time window. By default, a single 429 error rotates the key (`OMNIROUTE_ROTATE_429_THRESHOLD=1`), but this is configurable per status code. The `gateFor` function in [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) evaluates these rules against each failed request.

### How does OmniRoute handle rate limits differently from quota exhaustion?

Rate limits (typically 429) trigger **rotation or cooldown** based on configuration, while quota exhaustion may be detected through provider-specific error text classification. The `classifyErrorText` function in [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) parses error messages to distinguish transient rate limits from permanent quota depletion, applying different rotation policies accordingly.

### Can a single failing model disable an entire provider account?

No. For providers identified by `hasPerModelQuota` (Gemini, Codex), OmniRoute uses **per-model lockouts** via `lockModelIfPerModelQuota`. Only the specific model enters cooldown while other models on the same credential remain available. This is essential for providers that multiplex dozens of models through single API keys.

### What happens when all accounts for a provider are in cooldown?

The provider-wide circuit breaker in `isProviderInCooldown` returns `true`, causing the router in [`combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/combo.ts) to skip that provider entirely. Requests then fail over to alternative providers in the configured pool. If no providers are available, the gateway returns an appropriate error to the client rather than queueing indefinitely.