# How OmniRoute Handles Upstream 429 Errors with Exponential Backoff

> Learn how OmniRoute handles upstream 429 errors using per-credential exponential backoff for automatic recovery. Discover its robust error handling.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-20

---

**OmniRoute treats HTTP 429 errors as temporary, not fatal, and applies per-credential exponential backoff with automatic recovery once the cooldown expires.**

When building resilient LLM routing infrastructure, handling rate limits gracefully separates production-ready systems from brittle ones. In the [diegosouzapw/OmniRoute](https://github.com/diegosouzapw/OmniRoute) codebase, upstream 429 responses trigger a sophisticated **connection-cooldown mechanism** that preserves system throughput while respecting provider limits. This article examines exactly how OmniRoute implements exponential backoff for 429 errors, including detection, calculation, header parsing, and automatic recovery.

## Detecting 429 Errors in the Account Fallback Layer

Rate limit detection happens in [`src/sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/accountFallback.ts). The `checkFallbackError()` function examines HTTP response status codes and classifies 429 as a rate-limit error requiring cooldown handling.

```ts
// src/sse/services/accountFallback.ts – detection logic (simplified)
function checkFallbackError(response: Response, connection: ProviderConnection): ErrorClassification {
  if (response.status === 429) {
    return { type: 'RATE_LIMIT', retryable: true };
  }
  // ... other error types
}

```

Once classified, the error code forwards to the credential-management service for backoff processing. This separation of concerns keeps error detection decoupled from cooldown policy implementation.

## Marking Accounts Unavailable with Backoff Levels

The `markAccountUnavailable()` function in [`src/sse/services/auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/services/auth.ts) (lines 860–870) records the 429 error and establishes the cooldown period. It accepts the connection ID, HTTP status, error message, provider identifier, and optional model scope.

```ts
// Example from src/sse/services/auth.ts (lines 860‑870)
await auth.markAccountUnavailable(
  connectionId,            // the failing connection
  429,                     // HTTP status
  "Too many requests",     // error message
  provider,                // e.g. "openai"
  model,                   // optional model scope
);

```

This function increments a **backoff level** counter stored on the `ProviderConnection` record and computes a `rateLimitedUntil` timestamp. Higher backoff levels indicate repeated rate limiting on the same credential.

## Exponential Backoff Calculation

OmniRoute derives cooldown intervals from the global `COOLDOWN_MS` constant defined in [`src/sse/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/config/constants.ts). The exponential backoff formula follows standard practice:

```ts
// src/sse/config/constants.ts
export const COOLDOWN_MS = 5_000; // base 5 seconds

// src/sse/services/auth.ts – backoff calculation
const backoffMs = COOLDOWN_MS * 2 ** (connection.backoffLevel ?? 0);
const unavailableUntil = new Date(Date.now() + backoffMs).toISOString();

```

With a 5-second base, this produces:
- **Level 0**: 5 seconds (first 429)
- **Level 1**: 10 seconds
- **Level 2**: 20 seconds
- **Level 3**: 40 seconds

The `backoffLevel` property starts at 0 and increments with each successive 429 on the same connection. As noted in source comments around line 677 of [`auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/auth.ts), this pattern prevents hammering already-limited endpoints while allowing quick recovery for transient issues.

### Capping Maximum Backoff

Practical limits prevent excessive cooldown periods. The implementation uses `Math.min` constraints:

```ts
// auth.ts – cap calculation (simplified)
const cappedLevel = Math.min(40, (connection.backoffLevel || 0) * 8);

```

This ensures no credential remains unavailable indefinitely due to runaway backoff accumulation.

## Respecting Upstream Retry-After Headers

When providers include explicit guidance, OmniRoute prefers that over calculated backoff. The [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) implementation checks for `Retry-After` headers:

```ts
// src/sse/services/accountFallback.ts
if (upstreamHeaders.get('Retry-After')) {
  // Parse header and use exact value instead of exponential backoff
  const exactMs = parseRetryAfterHeader(...);
  rateLimitedUntil = new Date(Date.now() + exactMs).toISOString();
}

```

The header parser handles both:
- **Delay-seconds format**: `Retry-After: 120`
- **HTTP-date format**: `Retry-After: Wed, 21 Oct 2026 07:28:00 GMT`

This provider-aware flexibility ensures OmniRoute adapts to different API behaviors rather than applying uniform assumptions.

## Automatic Recovery and Lazy Decay

Once a cooldown period expires, OmniRoute automatically restores the connection without manual intervention. The credential selection routine checks `rateLimitedUntil` timestamps, and expired cooldowns trigger backoff level reset.

```ts
// Inside the credential‑selection routine (src/sse/services/auth.ts)
if (!allowRateLimitedConnections && isAccountUnavailable(conn.rateLimitedUntil)) {
  // Skip this connection; try the next one
  continue;
}

```

The auto-decay logic around lines 1439–1445 of [`auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/auth.ts) resets `backoffLevel` to 0 when `rateLimitedUntil` has passed, making the connection immediately eligible for new requests.

## Per-Connection and Per-Model Scope

The cooldown mechanism operates at two granularities:

1. **Connection-level**: The default behavior isolates rate limits to individual credentials. Other keys for the same provider remain usable, maintaining system capacity.

2. **Model-level**: When only a specific model triggers the 429, [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) can apply a **model-lockout** instead of full connection cooldown. This finer scope preserves access to non-limited models on the same credential.

This scope flexibility maximizes available routing options during partial rate limit scenarios.

## Summary

- **Detection**: `checkFallbackError()` in [`accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/accountFallback.ts) identifies 429 status codes
- **Backoff calculation**: `COOLDOWN_MS * 2 ** backoffLevel` with 5-second base and 40-level cap
- **Header respect**: `Retry-After` values override exponential calculation when present
- **Automatic recovery**: Expired `rateLimitedUntil` timestamps reset `backoffLevel` to 0
- **Isolated scope**: Cooldowns apply per-credential with optional per-model granularity

## Frequently Asked Questions

### How does OmniRoute differentiate between temporary and permanent errors?

OmniRoute classifies 429 as retryable through the `checkFallbackError()` return type. Fatal errors (authentication failures, invalid requests) bypass the cooldown mechanism entirely and trigger immediate credential rotation or request failure.

### Can I configure the base cooldown duration?

The `COOLDOWN_MS` constant in [`src/sse/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/config/constants.ts) controls the base value. Modifying this source constant changes all exponential backoff calculations system-wide. Runtime configuration per-provider is not exposed in the current implementation.

### What happens if all credentials for a provider enter cooldown?

When all connections for a provider become unavailable, OmniRoute's fallback logic exhausts the credential pool and propagates the 429 to the caller. This prevents indefinite request hanging while signaling genuine capacity constraints upstream.

### Does OmniRoute support jitter in the backoff calculation?

The analyzed source does not implement randomized jitter. Backoff intervals follow deterministic exponential growth: `base * 2^level`. Production deployments requiring jitter would need to modify the calculation in [`auth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/auth.ts) around the backoff computation logic.