# How to Configure Rate Limiting with Quota Trackers and Sliding Window Limiters in OmniRoute

> Learn to configure rate limiting in OmniRoute using Redis-backed sliding windows and quota trackers. Implement robust API protection with real-time monitoring and standardized responses.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-08-03

---

**OmniRoute implements a four-layer rate limiting architecture combining Redis-backed sliding windows, persistent database rate limits, real-time quota monitoring, and standardized HTTP 429 responses.**

This guide walks through configuring each component of OmniRoute's rate limiting system. The repository `diegosouzapw/OmniRoute` provides both **sliding window limiters** for request throttling and **quota trackers** for provider-specific usage monitoring, making it suitable for multi-tenant API gateways.

## Sliding-Window Rate Limiting: Redis or In-Memory

OmniRoute's core rate limiter lives in [`src/shared/utils/rateLimiter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimiter.ts). It supports **dual backends**: Redis for distributed deployments and an in-memory `Map` for single-node or test environments.

### Defining Rate Limit Rules

A **rule** is a `{ limit, window }` pair where `window` is expressed in seconds:

```typescript
const rules = [
  { limit: 100, window: 60 },      // 100 requests per minute
  { limit: 1_000, window: 3_600 }  // 1,000 requests per hour
];

```

### Using `checkRateLimit()`

The `checkRateLimit(identifier, rules)` function (lines 221-266) performs the actual enforcement:

```typescript
import { checkRateLimit } from "@/shared/utils/rateLimiter";

export async function handler(req: Request) {
  const apiKey = req.headers.get("x-api-key");
  const result = await checkRateLimit(apiKey ?? "anonymous", rules);
  
  if (!result.allowed) {
    return new Response(
      `Rate limit exceeded (window: ${result.failedWindow}s)`,
      { status: 429 }
    );
  }
  // Continue processing...
}

```

**Backend selection logic:**
- If `REDIS_URL` is configured, executes the `RATE_LIMIT_SCRIPT` Lua script atomically
- Otherwise, uses a per-process `Map` with `evictStaleRateLimitWindows()` for memory management

### Implementing Per-Route Rate Limits

Add custom rules to any API route. Here's a chat completions endpoint with tiered limits:

```typescript
// src/app/api/v1/chat/completions/route.ts
import { checkRateLimit } from "@/shared/utils/rateLimiter";

const CHAT_RULES = [
  { limit: 50, window: 60 },      // 50 req/min per API key
  { limit: 1_000, window: 86_400 } // 1,000 req/day
];

export async function POST(req: Request) {
  const apiKey = req.headers.get("x-api-key") ?? "guest";
  const rl = await checkRateLimit(apiKey, CHAT_RULES);
  
  if (!rl.allowed) {
    return new Response("Too many requests", { status: 429 });
  }
  // Handle chat request...
}

```

## Persistent Connection Rate Limiting

Unlike memory-only solutions, OmniRoute **persists rate-limit state per provider connection** in SQLite. This prevents OAuth token refreshes from clearing back-off timers.

### Core Functions in [`src/lib/db/providers/rateLimit.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/providers/rateLimit.ts)

| Function | Purpose |
|----------|---------|
| `markConnectionRateLimitedUntil(connectionId, retryAfterMs)` | Records when a connection can be reused |
| `isConnectionRateLimited(connectionId)` | Checks cooldown status during routing |
| `setConnectionRateLimitUntil(connectionId, timestamp)` | Direct timestamp assignment |
| `clearStaleCrashCooldowns()` | Cleanup utility for server startup |

### Handling Provider 429 Responses

When an upstream provider returns rate-limit errors, persist the back-off:

```typescript
import { markConnectionRateLimitedUntil } from "@/lib/db/providers/rateLimit";

function handleProvider429(connId: string, retryAfterMs: number) {
  // Stores in provider_connections.rate_limited_until
  markConnectionRateLimitedUntil(connId, retryAfterMs);
}

```

During request routing (in `open-sse/services/combo/`), `isConnectionRateLimited(connectionId)` filters out cooling-down accounts before selection.

### Startup Maintenance

Clear stale cooldowns on server bootstrap:

```typescript
// src/server/startup.ts
import { clearStaleCrashCooldowns } from "@/lib/db/providers/rateLimit";

export async function bootstrap() {
  const { cleared } = clearStaleCrashCooldowns();
  console.info(`[RateLimit] Cleared ${cleared} stale cooldowns`);
  // Continue startup...
}

```

## Quota Tracking: Pre-Flight and Monitor Services

OmniRoute's **quota system** tracks provider-specific usage limits (tokens, requests, cost) through three coordinated services.

### Quota Fetcher Registration

Register provider-specific fetchers in [`open-sse/services/quotaTrackersBatch.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/quotaTrackersBatch.ts) (lines 10-31):

```typescript
// Example pattern from quotaTrackersBatch.ts
registerAgentrouterQuotaFetcher();  // Registers fetcher for AgentRouter provider

```

Each fetcher returns:

```typescript
{
  used: number,
  total: number,
  percentUsed: number,
  resetAt: Date
}

```

### Enabling Session-Bound Quota Monitoring

Set `quotaMonitorEnabled: true` in `providerSpecificData`:

```typescript
// src/lib/db/providers/providerConfig.ts
export function enableQuotaMonitor(connectionId: string) {
  const db = getDbInstance();
  db.prepare(`
    UPDATE provider_connections
    SET providerSpecificData = json_set(
      providerSpecificData, 
      '$.quotaMonitorEnabled', 
      true
    )
    WHERE id = ?
  `).run(connectionId);
}

```

When active sessions use this connection, `startQuotaMonitor()` (invoked from [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts)) begins automatic polling.

### Monitor Behavior ([`quotaMonitor.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/quotaMonitor.ts))

The monitor service (lines 101-124) provides:

- **Adaptive polling**: Normal → Critical interval based on usage percentage
- **Alert suppression**: Deduplicated exhaustion warnings via `alertSuppression`
- **State management**: `MonitorState` tracks latest readings

Retrieve snapshots for UI display:

```typescript
import { getQuotaMonitorSnapshots, getQuotaMonitorSummary } from "@/open-sse/services/quotaMonitor";

const health = getQuotaMonitorSummary(sessionId);

```

## API-Level Rate-Limited Responses

Standardize 429 responses using [`src/app/api/v1/_shared/rateLimit.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/_shared/rateLimit.ts):

```typescript
import { rateLimitedProviderResponse } from "@/app/api/v1/_shared/rateLimit";

// When all provider accounts are exhausted
if (allRateLimited) {
  const credentials: RateLimitedCredentials = { 
    allRateLimited: true, 
    retryAfter: retryAfterMs 
  };
  return rateLimitedProviderResponse("openai", credentials);
}

```

This helper returns properly formatted HTTP 429 or 509 responses with structured JSON bodies for client handling.

## Rate Limit Manager and Semaphore

Additional coordination utilities in the `open-sse/services/` layer:

- **[`rateLimitManager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rateLimitManager.ts)** – Core coordination logic for multi-provider scenarios
- **[`rateLimitSemaphore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rateLimitSemaphore.ts)** – MCP-side rate-limit semaphore for concurrent request control

These integrate with the sliding window limiter for **cross-cutting rate control** across provider pools.

## Configuration Checklist

| Layer | Configuration Required | Key File |
|-------|------------------------|----------|
| Sliding window | `REDIS_URL` env var (optional) | [`src/shared/utils/rateLimiter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimiter.ts) |
| Connection persistence | SQLite schema auto-migrated | [`src/lib/db/providers/rateLimit.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/providers/rateLimit.ts) |
| Quota monitoring | `quotaMonitorEnabled: true` in connection config | [`open-sse/services/quotaMonitor.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/quotaMonitor.ts) |
| Provider fetchers | Register in batch file | [`open-sse/services/quotaTrackersBatch.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/quotaTrackersBatch.ts) |

## Summary

- **`checkRateLimit()`** in [`src/shared/utils/rateLimiter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimiter.ts) enforces sliding-window limits with Redis or in-memory fallback
- **Connection-level rate limits** persist in SQLite via `markConnectionRateLimitedUntil()` to survive OAuth refreshes
- **Quota trackers** combine pre-flight fetchers with optional session polling for real-time exhaustion alerts
- **`rateLimitedProviderResponse()`** standardizes HTTP 429 responses across all API routes
- Enable monitoring by setting `quotaMonitorEnabled: true` in provider connection configuration

## Frequently Asked Questions

### What's the difference between rate limiting and quota tracking in OmniRoute?

**Rate limiting** counts requests and rejects excess traffic with HTTP 429, implemented via `checkRateLimit()` using sliding windows. **Quota tracking** monitors provider-specific resources (tokens, cost, API calls) and predicts exhaustion before limits are hit. Rate limiting is enforced; quota tracking is monitored and alerted.

### Can I use OmniRoute's rate limiter without Redis?

Yes. When `REDIS_URL` is unset, [`src/shared/utils/rateLimiter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimiter.ts) falls back to a per-process `Map` with automatic eviction of stale windows via `evictStaleRateLimitWindows()`. This works for single-node deployments but won't share state across server instances.

### How does connection-level rate limiting prevent OAuth token issues?

The `markConnectionRateLimitedUntil()` function in [`src/lib/db/providers/rateLimit.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/providers/rateLimit.ts) stores the back-off timestamp in the `provider_connections` table. Unlike memory-only stores, this survives token refreshes that recreate connection objects, ensuring consistent cool-down enforcement.

### When should I enable quota monitoring for a provider?

Enable `quotaMonitorEnabled` when the provider exposes usage APIs and you need **predictive exhaustion handling**. The monitor in [`open-sse/services/quotaMonitor.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/quotaMonitor.ts) polls periodically, emits alerts before hard limits, and adjusts polling frequency based on urgency—ideal for cost-controlled or token-limited APIs.