# OmniRoute's Three-Layer Resilience System: Circuit Breaker, Fallback, and Emergency Explained

> Discover OmniRoute's three-layer resilience system: circuit breaker, fallback, and emergency. Protect your requests with throttling, isolation, and guaranteed responses. Learn more now!

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-07-01

---

**OmniRoute protects every request with a three-layer resilience architecture that throttles connection cooldowns, isolates failing providers via circuit breakers, and guarantees responses through an emergency fallback when all targets are exhausted.**

The diegosouzapw/OmniRoute repository implements a fault-tolerant routing pipeline designed to prevent cascading failures, safeguard upstream credentials, and maintain availability even when AI providers exhaust quotas or return errors. By combining request-queue throttling, provider circuit breakers, and a credential-isolated emergency fallback, the system ensures that a single point of failure never degrades the entire user experience.

## Layer 1: Connection Cooldown and Request Throttling

The first layer prevents a flood of retries from overwhelming a provider by implementing **per-connection rate limiting**. In [`src/lib/resilience/services.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/services.ts), a **Bottleneck** queue throttles calls for each connection based on the `resilienceSettings.connectionCooldownMs` value defined in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts).

When a provider returns a **429** (rate-limit) status, the connection enters a cooldown period. New requests targeting a cooled-down connection are immediately skipped, and the router attempts the next available provider. This protects upstream services from being hammered during transient overloads while preserving client resources.

## Layer 2: Provider Circuit Breaker

The second layer stops repeated failing calls to a faulty provider from consuming resources and leaking credentials. Each provider profile in [`src/shared/constants/providerProfiles.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providerProfiles.ts) defines two critical thresholds: `circuitBreakerThreshold` (maximum consecutive failures) and `circuitBreakerReset` (cool-down time in milliseconds).

The generic **CircuitBreaker** class in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) tracks failures and state transitions between **CLOSED** (normal operation) and **OPEN** (failing fast). When failures exceed the threshold, the circuit opens and refuses further calls until the reset timeout expires. The `classifyFailKind` function marks these failures as `circuit-open`, signaling the routing engine to bypass the provider instantly.

## Layer 3: Emergency Fallback

The final layer guarantees a response when **all** targets are exhausted due to budget limits, quota restrictions, or open circuits. The **EmergencyFallback** service in [`open-sse/services/emergencyFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/emergencyFallback.ts) activates when the routing layer detects a `budget-exhausted` or `circuit-open` error.

Controlled by the `EMERGENCY_FALLBACK_FLAG_KEY` (exposed in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts)), this layer routes the request to a free fallback model—by default `openai/gpt-oss-120b`. The `runtimeOptions.emergencyFallbackTried` flag ensures the fallback is invoked **only once per request**, and the implementation explicitly prevents credential leakage by never forwarding the original provider’s API keys or tokens.

## How the Layers Interact

The resilience system operates as a cascading filter during the request lifecycle:

1. **Incoming request** enters through [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts).
2. **Rate-limit queue** (`withRateLimit`) attempts the first target. If the provider is in connection cooldown, the queue skips it immediately.
3. If the provider returns a **429** or a **circuit-open** pattern, `classifyFailKind` in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) records the failure and triggers the circuit breaker if thresholds are met.
4. The routing engine ([`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) via `handleComboChat`) selects the next candidate provider.
5. When **all** targets report `budget-exhausted` or `circuit-open`, the **EmergencyFallback** service checks the feature flag and routes to the free model without exposing original credentials.
6. The response streams back to the client. If the emergency model also fails, the system returns the original error to ensure transparency.

## Configuring Resilience Settings

### Resetting Circuit Breakers via API

To manually reset all provider circuits after an outage, use the management API endpoint defined in [`src/app/api/resilience/reset/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/reset/route.ts):

```bash
curl -X POST http://localhost:20128/api/resilience/reset \
     -H "Authorization: Bearer <MANAGEMENT_TOKEN>"

```

This invokes `resetAllCircuitBreakers()` from [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), clearing failure counts and returning all circuits to the CLOSED state.

### Enabling Emergency Fallback at Runtime

Toggle the emergency fallback feature without restarting the service:

```typescript
import { setFeatureFlag } from "@omniroute/open-sse/utils/featureFlags";

await setFeatureFlag("EMERGENCY_FALLBACK", true);

```

The [`open-sse/services/emergencyFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/emergencyFallback.ts) module caches this flag value in `emergencyFallbackFlagCache` and evaluates it on every request to determine if the free fallback model should be used.

### Adjusting Circuit-Breaker Thresholds

Modify provider-specific resilience profiles via the PATCH endpoint in [`src/app/api/resilience/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/route.ts):

```bash
curl -X PATCH http://localhost:20128/api/resilience \
     -H "Authorization: Bearer <MANAGEMENT_TOKEN>" \
     -H "Content-Type: application/json" \
     -d '{
           "providerBreaker": {
             "oauth": { "failureThreshold": 10, "resetTimeoutMs": 120000 }
           }
         }'

```

This updates the persisted `ResilienceSettings` record, which the circuit breaker reads via `PROVIDER_PROFILES` in [`src/shared/constants/providerProfiles.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providerProfiles.ts).

## Summary

- **Connection Cooldown** ([`src/lib/resilience/services.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/services.ts)) prevents provider overload by throttling requests after 429 responses using `resilienceSettings.connectionCooldownMs`.
- **Circuit Breaker** ([`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts)) isolates flaky providers after configurable failure thresholds, protecting credentials and resources.
- **Emergency Fallback** ([`open-sse/services/emergencyFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/emergencyFallback.ts)) guarantees service continuity with a free model when all providers are exhausted, ensuring `emergencyFallbackTried` prevents infinite loops.
- All layers are configurable via the Resilience API ([`src/app/api/resilience/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/route.ts)) and respect the `EMERGENCY_FALLBACK_FLAG_KEY` feature flag.

## Frequently Asked Questions

### How does OmniRoute prevent credential leakage during provider failures?

The emergency fallback implementation in [`open-sse/services/emergencyFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/emergencyFallback.ts) explicitly strips original provider credentials before routing to the fallback model `openai/gpt-oss-120b`. As noted in the credential-leak guard comments within [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts), the system never forwards API keys or tokens from exhausted providers to the emergency service, ensuring that authentication secrets remain isolated to their intended targets.

### What triggers the circuit breaker to open?

The circuit breaker opens when consecutive failures exceed the `circuitBreakerThreshold` defined in [`src/shared/constants/providerProfiles.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/providerProfiles.ts). Failures are classified by the `classifyFailKind` function in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), which tracks 5xx errors, malformed responses, and authentication failures. Once the threshold is reached, the circuit transitions to OPEN and remains in that state for the duration specified by `circuitBreakerReset` (in milliseconds).

### Can I customize the emergency fallback model?

Yes. While the default emergency model is `openai/gpt-oss-120b`, you can configure alternative endpoints by modifying the fallback resolver logic in [`open-sse/services/emergencyFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/emergencyFallback.ts). Ensure that any custom model does not require the original provider's credentials, as the system explicitly disables credential forwarding to prevent leakage during fallback execution.

### How do I reset a tripped circuit breaker programmatically?

Use the `resetAllCircuitBreakers()` utility exported from [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), which is exposed via the POST endpoint at [`src/app/api/resilience/reset/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/reset/route.ts). This clears the failure counters for all providers, immediately transitioning their circuits from OPEN to CLOSED and restoring normal traffic flow to previously isolated targets.