# How to Configure Provider-Specific Rate Limits and Circuit Breaker Thresholds in OmniRoute

> Learn to configure provider-specific rate limits and circuit breaker thresholds in OmniRoute using the /api/resilience endpoint. Optimize your API resilience with custom settings.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-27

---

**OmniRoute lets you configure provider-specific rate limits and circuit breaker thresholds through the `/api/resilience` PATCH endpoint, with defaults defined in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) and runtime enforcement handled by [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) and [`src/shared/utils/rateLimitSemaphore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimitSemaphore.ts).**

OmniRoute is an open-source LLM routing layer that isolates upstream failures through per-provider resilience controls. You can configure provider-specific rate limits and circuit breaker thresholds to prevent cascading outages and honor provider throttling without redeploying the service. The system stores type-based defaults and accepts live overrides through a dedicated REST API.

## Default Provider Resilience Profiles

OmniRoute defines default thresholds per authentication type in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts). The `PROVIDER_PROFILES` object supplies fallback values when a connection record lacks its own configuration:

```ts
// src/lib/resilience/settings.ts
export const PROVIDER_PROFILES = {
  oauth: {
    circuitBreakerThreshold: 5,   // fail after 5 consecutive errors
    circuitBreakerReset: 30_000, // stay open 30 s before half-open trial
  },
  apikey: {
    circuitBreakerThreshold: 10,
    circuitBreakerReset: 60_000,
  },
};

```

These defaults differentiate **OAuth** connections, which use a lower failure tolerance, from **API-key** connections, which tolerate more errors before opening the circuit. If a provider record does not specify `circuitBreakerThreshold` or `circuitBreakerReset`, the routing layer falls back to the matching profile above.

## Overriding Per-Provider Thresholds via the Resilience API

You can inspect and mutate thresholds at runtime through the `/api/resilience` endpoint. A **GET** request returns the current global and per-provider settings, while a **PATCH** request merges a partial payload into the live configuration.

The endpoint lives in [`src/app/api/resilience/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/route.ts). When it receives a PATCH, it merges the incoming JSON and immediately refreshes the in-memory utilities:

```ts
// src/app/api/resilience/route.ts
import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";

export async function PATCH(req) {
  const body = await req.json();
  // merge logic → SETTINGS.providerBreaker[provider] = …
  // refresh in-memory circuit-breaker caches
  const breaker = getCircuitBreaker();
  breaker.updateProviderSettings(providerId, body.providerBreaker[providerId]);
}

```

A minimal payload adjusts both circuit-breaker and rate-limit behavior for a single provider:

```json
{
  "providerBreaker": {
    "my-provider-id": {
      "failureThreshold": 3,
      "resetTimeoutMs": 15_000
    }
  },
  "rateLimit429": {
    "threshold": 1,
    "windowMs": 120_000,
    "cooldownMs": 10_000
  }
}

```

The same endpoint accepts `rateLimit429` configuration, which propagates to [`src/shared/utils/rateLimitSemaphore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimitSemaphore.ts). Changes take effect immediately without restarting the service.

To update a provider's circuit breaker from the command line:

```bash
curl -X PATCH https://my.omniroute.local/api/resilience \
  -H "Content-Type: application/json" \
  -d '{
    "providerBreaker": {
      "openai": { "failureThreshold": 4, "resetTimeoutMs": 20000 }
    }
  }'

```

## Runtime Interaction Between Circuit Breakers and Rate Limits

Before every upstream request, OmniRoute evaluates both controls. The check occurs in [`src/sse/handlers/chatHelpers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chatHelpers.ts) (called from [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts)):

```ts
// src/sse/handlers/chatHelpers.ts
if (circuitBreaker.isOpen(providerId)) {
  // short-circuit: return a "circuit-open" error without upstream call
}
if (rateLimitSemaphore.isRateLimited(modelId)) {
  // short-circuit: return a "rate-limited" error with Retry-After header
}

```

**Rate-limit cooldown** blocks a provider after receiving an HTTP 429 or any rate-limited response. The semaphore stores a `rateLimitedUntil` timestamp per model and queues excess calls. **Circuit-breaker** logic counts consecutive failures—such as network errors, 5xx responses, or explicit "circuit open" messages—and opens the circuit after the configured threshold. While open, the provider is bypassed entirely until the reset timeout expires.

The circuit breaker takes precedence over the rate-limit check. In `classifyFailKind()`, a message containing both "429" and "circuit open" is classified as a circuit-open event, a behavior validated by [`tests/unit/ui/comboFlowModel.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/ui/comboFlowModel.test.ts).

## Persistence and Observability

Rate-limit timestamps are stored as `rateLimitedUntil` values in the `connections` table managed by [`src/lib/db/connections.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/connections.ts). Circuit-breaker state is maintained in an in-memory map populated at startup and refreshed on every successful request. Stale cooldowns and failure counters are cleared automatically when traffic resumes.

For real-time visibility, the monitoring API exposes the current circuit state via [`src/lib/monitoring/providerHealthMatrix.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthMatrix.ts). Query the health endpoint to inspect provider status:

```bash
curl https://my.omniroute.local/api/monitoring/health

```

The response includes a per-provider `circuitBreaker` field:

```json
{
  "provider": "openai",
  "circuitBreaker": { "state": "CLOSED", "failureCount": 0 }
}

```

You can also force a rate-limit cooldown programmatically for testing:

```ts
import { rateLimitSemaphore } from "@/shared/utils/rateLimitSemaphore";

await rateLimitSemaphore.markRateLimited("gpt-4", 30_000); // 30 s cooldown

```

## Summary

- **Default profiles** in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) supply OAuth and API-key circuit-breaker baselines.
- **Live overrides** are applied through PATCH `/api/resilience`, which updates `circuitBreaker` and `rateLimitSemaphore` in memory without a restart.
- **Request guards** in [`src/sse/handlers/chatHelpers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chatHelpers.ts) check the circuit breaker first, then the rate-limit semaphore, with circuit state taking precedence.
- **Observability** is available through `/api/monitoring/health`, backed by [`src/lib/monitoring/providerHealthMatrix.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthMatrix.ts).

## Frequently Asked Questions

### How do I change the default circuit-breaker threshold for all OAuth providers?

Edit the `oauth` block inside `PROVIDER_PROFILES` in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) and redeploy. Any connection record without its own threshold will inherit the new default value.

### What happens when both rate-limit and circuit-breaker conditions are met?

The circuit breaker wins. According to the `classifyFailKind()` logic tested in [`tests/unit/ui/comboFlowModel.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/ui/comboFlowModel.test.ts), a response payload that contains both "429" and "circuit open" signals is treated as a circuit-open event, and the request is short-circuited before the rate-limit check.

### Does updating `/api/resilience` require a server restart?

No. The PATCH handler in [`src/app/api/resilience/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/route.ts) merges the incoming payload into the live configuration and calls `breaker.updateProviderSettings()` directly, so in-memory state reflects the new thresholds immediately.

### Where is rate-limit cooldown data stored?

The `rateLimitedUntil` timestamp for each model is persisted in the `connections` table via [`src/lib/db/connections.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/connections.ts), while the active semaphore state is held in memory inside [`src/shared/utils/rateLimitSemaphore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/rateLimitSemaphore.ts). Successful requests automatically clear stale cooldowns.