# How the Provider Cooldown Tracker Prevents 429 Rate Limit Errors in OmniRoute

> OmniRoute's provider cooldown tracker stops 429 rate limit errors. Learn how it pauses providers until cooldowns expire, ensuring smooth routing.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-01

---

**OmniRoute's provider cooldown tracker prevents 429 errors by storing a `rateLimitedUntil` deadline for each provider and account that hits a rate limit, causing the router to skip that provider entirely until the cooldown expires.**

The provider cooldown tracker in the open-source OmniRoute project (`diegosouzapw/OmniRoute`) is a lightweight, cross-request service designed to eliminate aggressive retry storms against upstream APIs. When a provider returns an HTTP 429 response, the tracker records a cooldown deadline that subsequent routing decisions consult before dispatching any new requests.

## Core Mechanism of the Provider Cooldown Tracker

The tracker operates as a centralized state manager that makes unified decisions across all routing components. It exposes three primary functions for managing cooldown state: `recordProviderCooldown`, `isProviderInCooldown`, and `getRemainingCooldownMs`.

### Recording Rate Limit Events

When a provider responds with a 429 status, the system immediately records the cooldown using the `recordProviderCooldown` function defined in [`open-sse/services/providerCooldownTracker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/providerCooldownTracker.ts). This function accepts the provider name, account identifier, and the `retryAfterMs` duration to calculate the `rateLimitedUntil` timestamp.

```typescript
// Record a 429 for a provider/account
import { recordProviderCooldown } from '@/open-sse/services/providerCooldownTracker';

recordProviderCooldown('openai', 'account-123', /* retryAfterMs */ 30_000);

```

This stores a deadline representing when the provider will become available again, protecting the upstream service from immediate subsequent requests.

### Checking Cooldown State Before Requests

Before dispatching any request, the router consults `isProviderInCooldown` to determine if the deadline is still in the future. If active, the router retrieves the remaining wait time via `getRemainingCooldownMs` and treats the provider as unavailable.

```typescript
// Later, before routing a request:
import { isProviderInCooldown, getRemainingCooldownMs } from '@/open-sse/services/providerCooldownTracker';

if (isProviderInCooldown('openai')) {
  const wait = getRemainingCooldownMs('openai'); // e.g. 28 000 ms
  // Skip this provider, try the next one in the combo
}

```

This short-circuit behavior ensures that requests never hit a provider that is known to be rate-limited, eliminating redundant 429 responses and maintaining low overall latency.

## Integration with Combo Routing

The cooldown tracker integrates directly with OmniRoute's combo routing logic in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts). During combo execution, the system iterates through target providers and actively filters out any that are currently in cooldown.

```typescript
// Combo routing snippet (simplified)
import { recordProviderCooldown, isProviderInCooldown } from '@/open-sse/services/providerCooldownTracker';

async function handleComboChat(combo, request) {
  for (const target of combo.targets) {
    if (isProviderInCooldown(target.provider)) continue;   // avoid 429‑prone provider
    const result = await dispatchToProvider(target, request);
    if (result.status === 429) {
      // Store cooldown and move on to the next target
      recordProviderCooldown(target.provider, target.account, result.retryAfterMs);
      continue;
    }
    return result; // success
  }
  throw new Error('All combo targets are in cooldown');
}

```

This integration ensures that the fallback chain automatically excludes rate-limited providers, allowing traffic to flow immediately to healthy alternatives without wasting time on doomed requests.

## Housekeeping and Cleanup

The tracker performs automatic maintenance to prevent memory leaks and handle server restarts. The `cleanupExpiredCooldownEntries` function removes entries whose deadlines have passed, while `clearStaleCrashCooldowns` prunes stale cooldowns on startup to recover from unexpected crashes.

Additionally, the health monitoring system in [`open-sse/services/webSessionPoolHealth.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/webSessionPoolHealth.ts) uses the tracker to surface "cooldown" badges in the dashboard UI, giving operators real-time visibility into which providers are currently throttled.

## Summary

- **Centralized state**: The provider cooldown tracker maintains `rateLimitedUntil` deadlines in [`open-sse/services/providerCooldownTracker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/providerCooldownTracker.ts), offering a single source of truth for rate-limit status across the application.
- **Short-circuit protection**: Functions `isProviderInCooldown` and `getRemainingCooldownMs` prevent requests from reaching providers that will return 429 errors, reducing latency and protecting upstream quotas.
- **Automatic fallback**: The combo routing system in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) integrates the tracker to skip cooldown providers and immediately attempt the next target in the chain.
- **Maintenance utilities**: `cleanupExpiredCooldownEntries` and `clearStaleCrashCooldowns` ensure the cooldown map remains clean across application lifecycles.

## Frequently Asked Questions

### What happens when a provider returns a 429 error?

When a provider returns HTTP 429, the system calls `recordProviderCooldown` with the provider name, account ID, and retry-after duration. This stores a deadline timestamp after which the provider becomes eligible for new requests. Until that deadline passes, `isProviderInCooldown` returns true, causing the router to skip that provider entirely.

### How does the cooldown tracker integrate with combo routing?

The combo routing logic in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) imports the cooldown API and checks each target via `isProviderInCooldown` before dispatching. If a target is in cooldown, the loop continues to the next target. If a request returns 429 during execution, the system records the cooldown immediately and continues to the next fallback target.

### Where is the cooldown state stored?

The cooldown state is managed within the provider cooldown tracker service located at [`open-sse/services/providerCooldownTracker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/providerCooldownTracker.ts). This cross-request service maintains the `rateLimitedUntil` timestamps in memory, making the state available to all routing components, health monitors, and UI renderers without requiring external database calls.

### How does the system handle stale cooldown entries?

The tracker provides `cleanupExpiredCooldownEntries` to remove expired deadlines during runtime, and `clearStaleCrashCooldowns` to prune entries on startup. This ensures that temporary rate limits do not persist indefinitely after application restarts or crashes, maintaining accurate availability status for all providers.