# How OmniRoute Handles Retries with Backoff for Upstream Requests

> Learn how OmniRoute handles retries with backoff for upstream requests. Discover its automatic doubling delay, Retry-After header support, and configurable limits.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-28

---

**OmniRoute implements a centralized retry mechanism using exponential backoff through the `withRetry` utility in [`src/lib/proxySubscription/fetchRetry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/fetchRetry.ts), automatically spacing retry attempts by doubling the delay after each transient failure while respecting provider `Retry-After` headers and configurable maximum ceilings.**

OmniRoute treats every call to upstream LLM providers as a potentially transient operation requiring resilient fault tolerance. The framework automatically retries failed requests using an exponential backoff strategy that is centralized, configurable, and applied consistently across proxy subscriptions, token refreshes, and circuit breakers according to the `diegosouzapw/OmniRoute` source code.

## Core Retry Architecture with `withRetry`

The foundation of OmniRoute's resilience strategy lives in [`src/lib/proxySubscription/fetchRetry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/fetchRetry.ts), which exports the generic `withRetry<T>()` function. This utility wraps any async operation, detects transient failures, and schedules subsequent attempts using an exponential delay calculation.

Rather than scattering retry logic throughout the codebase, OmniRoute centralizes the implementation so that proxy subscriptions, token health checks, and circuit breakers all invoke the same backoff algorithm. The function accepts a callback returning a Promise and automatically determines whether an error warrants a retry based on status codes and network conditions.

### Detecting Transient Failures

The retry logic classifies specific error conditions as transient and therefore eligible for backoff. According to the implementation in [`src/lib/proxySubscription/fetchRetry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/fetchRetry.ts), the system retries on HTTP status codes **502**, **503**, **504**, and **429**, plus network-level errors that indicate temporary unavailability. Non-transient errors fail immediately without consuming retry attempts.

## Exponential Backoff Algorithm

OmniRoute calculates retry delays using a standard exponential backoff formula with configurable guardrails. The algorithm executes these steps for every retry attempt:

1. **Classify the error** – Verify the failure is transient (5xx, 429, or network timeout).
2. **Determine base delay** – Use the provider's `minRetryCooldownMs` (default **500 ms**) or extract a `Retry-After` header value when present.
3. **Apply exponential multiplier** – Calculate `delay = baseDelay * Math.pow(2, attempt - 1)`, doubling the wait time with each attempt.
4. **Enforce ceiling** – Clamp the result against `maxBackoffMs` to prevent runaway wait times.
5. **Schedule execution** – Pause for the computed duration before invoking the next attempt, up to `maxRetries` (typically **3–5**).

If the upstream response includes a `Retry-After` header, OmniRoute honors this value over the calculated exponential delay. As implemented in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) (lines 728–755), the header value wins unconditionally, allowing the system to respect the provider's explicit throttling window.

### Connection-Level Cooldown Calculations

For per-connection failure tracking, [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) provides `calculateExponentialBackoffCooldown()`. This helper computes delays using `minRetryCooldownMs * 2^(failures - 1)` and respects the `maxCooldownMs` threshold defined in [`src/lib/resilience/settings/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings/types.ts). This ensures individual connection failures do not impact the global retry policy while still preventing thundering herds.

## Implementation Across the Request Lifecycle

OmniRoute wires the retry utility into several critical paths to ensure consistent behavior regardless of which upstream provider is being contacted.

### Proxy Subscription Handling

The subscription service in [`src/lib/proxySubscription/subscriptionService.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/subscriptionService.ts) wraps all upstream fetches using `withRetry()`. At line 379, the service passes its `doSafeFetch` function into the retry wrapper, ensuring that streaming subscriptions benefit from automatic recovery without dropping client connections during transient provider outages.

### Token Refresh Operations

Credential refreshes use the same exponential backoff strategy to avoid hammering authentication endpoints. The [`open-sse/services/tokenRefresh/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/tokenRefresh/circuitBreaker.ts) module exports `refreshWithRetry()`, which applies the backoff logic when fetching new tokens. Additionally, [`src/lib/tokenHealthCheck.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/tokenHealthCheck.ts) implements retry policies specifically for background token validation tasks.

### Provider Circuit Breakers

At the provider level, [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) tracks consecutive failures and transitions between **CLOSED**, **OPEN**, and **HALF-OPEN** states. When the breaker enters the **OPEN** state, it applies an exponential multiplier (default **×16**) to the reset timeout, effectively backing off the entire provider rather than individual requests. This prevents OmniRoute from repeatedly attempting requests to a completely unavailable upstream service.

## Configuration and Customization

All backoff parameters are centralized in [`src/lib/resilience/settings/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings/types.ts), allowing operators to adjust defaults globally or override values per provider via [`src/shared/constants/providers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providers.ts). Key configuration options include:

- `minRetryCooldownMs` – Base delay for the first retry (default 500 ms)
- `maxBackoffMs` – Hard ceiling for any delay calculation
- `maxRetries` – Maximum number of attempts before hard failure

## Code Examples

### Wrapping a Subscription Fetch

```typescript
import { withRetry } from "@/lib/proxySubscription/fetchRetry";

async function fetchData(url: string, headers: HeadersInit) {
  return withRetry(() => doSafeFetch(url, headers));
}

```

*Source: [`src/lib/proxySubscription/subscriptionService.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/subscriptionService.ts) (line 379)*

### Token Refresh with Backoff

```typescript
import { refreshWithRetry } from "open-sse/services/tokenRefresh/circuitBreaker";

const refreshed = await refreshWithRetry(providerId, async () => {
  return await fetchTokenEndpoint(...);
});

```

*Source: [`open-sse/services/tokenRefresh/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/tokenRefresh/circuitBreaker.ts) (line 114)*

### Manual Backoff Calculation

```typescript
import { calculateExponentialBackoffCooldown } from "open-sse/services/accountFallback";

const cooldown = calculateExponentialBackoffCooldown({
  failures: 3,
  minRetryCooldownMs: 500,
  maxCooldownMs: 30_000,
});

```

*Source: [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) (line 1546)*

### Key Implementation Files

- **[`src/lib/proxySubscription/fetchRetry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/fetchRetry.ts)** – Core `withRetry` implementation and exponential backoff logic
- **[`src/lib/proxySubscription/subscriptionService.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/subscriptionService.ts)** – Integration point for streaming subscriptions
- **[`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts)** – Per-connection cooldown calculations and `Retry-After` header handling
- **[`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)** – Provider-level circuit breaker with exponential multipliers
- **[`src/lib/resilience/settings/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings/types.ts)** – Configuration types for backoff min/max values
- **[`open-sse/services/tokenRefresh/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/tokenRefresh/circuitBreaker.ts)** – Token refresh retry wrapper
- **[`src/lib/tokenHealthCheck.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/tokenHealthCheck.ts)** – Background token validation retry strategy

## Summary

- OmniRoute centralizes retry logic in the `withRetry` utility within [`src/lib/proxySubscription/fetchRetry.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/proxySubscription/fetchRetry.ts), ensuring consistent behavior across proxy subscriptions and token operations.
- The exponential backoff algorithm doubles the delay after each failure using `baseDelay * 2^(attempt-1)` while respecting configurable minimums and maximums defined in [`src/lib/resilience/settings/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings/types.ts).
- Transient error detection covers HTTP 5xx status codes, 429 rate limits, and network failures, immediately failing on permanent errors to conserve resources.
- The system respects upstream `Retry-After` headers when present, overriding the exponential calculation to comply with provider-specific throttling windows.
- Circuit breakers in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) provide provider-level protection using exponential multipliers, while connection-level backoff in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) isolates failures to specific upstream links.

## Frequently Asked Questions

### What HTTP status codes trigger a retry in OmniRoute?

OmniRoute retries requests that return **502 Bad Gateway**, **503 Service Unavailable**, **504 Gateway Timeout**, or **429 Too Many Requests**, along with network-level errors indicating temporary unavailability. Permanent errors such as 400 Bad Request or 401 Unauthorized fail immediately without retry attempts, as these indicate client-side issues that will not resolve with time.

### How does OmniRoute handle rate-limiting headers from upstream providers?

When an upstream response includes a `Retry-After` header, OmniRoute extracts this value and uses it as the delay duration for the next retry attempt, bypassing the exponential calculation. This logic is implemented in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) (lines 728–755), ensuring the framework respects the provider's explicit throttling instructions rather than guessing the appropriate backoff timing.

### Can I customize the retry and backoff behavior for specific providers?

Yes. Omniroute exposes retry configuration through [`src/lib/resilience/settings/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings/types.ts), where you can define `minRetryCooldownMs`, `maxBackoffMs`, and `maxRetries` values. Per-provider overrides can be specified in [`src/shared/constants/providers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providers.ts), allowing different upstream services to have unique retry policies based on their reliability characteristics or contractual rate limits.

### What is the difference between connection-level and provider-level backoff?

Connection-level backoff, calculated by `calculateExponentialBackoffCooldown()` in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts), tracks failures for individual upstream connections and applies exponential delays using `minRetryCooldownMs * 2^(failures-1)`. Provider-level backoff, managed by [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), tracks failures across all connections to a specific provider and applies an exponential multiplier (default ×16) to the circuit reset timeout when the breaker opens, protecting the entire service from repeatedly attempting requests to a completely failed provider.