# How OmniRoute Implements Provider Fallback Mechanism: Circuit Breaker and Emergency Fallback

> Learn how OmniRoute implements provider fallback with circuit breakers and emergency fallback for high availability. Protect your service from provider failures.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-30

---

**OmniRoute ensures high availability through a three-tier resilience strategy that combines declarative fallback chains, per-provider circuit breakers, and a final emergency fallback to handle provider failures automatically.**

The provider fallback mechanism in OmniRoute guarantees that LLM requests succeed even when primary providers fail. By combining a declarative policy engine with stateful circuit breakers and a last-resort emergency provider, the routing layer automatically isolates unhealthy endpoints and redirects traffic to viable alternatives. This article examines the implementation details found in the diegosouzapw/OmniRoute repository, breaking down exactly how the system handles degradation and recovery.

## Declarative Fallback Chains

The foundation of OmniRoute's provider fallback mechanism is the `FallbackEntry` chain stored in [`src/domain/fallbackPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/fallbackPolicy.ts). Each model can have a priority-ordered list of alternate providers that the engine consults when the primary fails.

### Registering Provider Chains

Fallback chains are registered declaratively using the `registerFallback()` function and persisted in SQLite with an in-memory `Map` cache:

```typescript
// src/domain/fallbackPolicy.ts
registerFallback("gpt-4o", [
  { provider: "openai",    priority: 0 },
  { provider: "anthropic", priority: 1 },
  { provider: "groq",      priority: 2, enabled: false }, // disabled entry
]);

```

Each entry includes a `priority` integer (lower is higher priority) and an optional `enabled` boolean to temporarily remove providers from rotation without deleting the configuration.

### Resolving Fallback Targets

During request processing, the policy engine calls `resolveFallbackChain(model, excludedProviders)` to retrieve enabled providers sorted by priority:

```typescript
// src/domain/policyEngine.ts
const fallbackChain = resolveFallbackChain(model);

```

This function filters out disabled entries and excluded providers (those with open circuits), returning only viable alternatives for the request handler to attempt.

## Circuit Breaker State Machine

OmniRoute implements a sophisticated circuit breaker in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) that tracks per-provider health and prevents cascading failures by short-circuiting requests to unhealthy endpoints.

### Per-Provider Failure Tracking

Each provider receives a named circuit breaker via `getCircuitBreaker(name, options)`. The breaker maintains a state machine with five states: **CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED**.

```typescript
// src/sse/handlers/chat.ts
const breaker = getCircuitBreaker(provider, {
  failureThreshold: providerProfile.failureThreshold,
  resetTimeout: providerProfile.resetTimeoutMs,
  isFailure: (e) => !isLocalStreamLifecycleError(e), // ignore local stream errors
  classifyError,
  cooldownByKind: { 
    rate_limit: 60_000, 
    quota_exhausted: 3_600_000 
  },
});

```

The breaker classifies errors by kind, applying specific cooldown periods for rate limits (60 seconds) and quota exhaustion (1 hour). Local stream lifecycle errors are explicitly excluded from failure counting to avoid penalizing providers for client-side connection issues.

### State Transitions and Back-off

When failures exceed the configured threshold, the breaker transitions to `OPEN`, rejecting all requests until the reset timeout expires. After the timeout, it enters `HALF_OPEN` to probe for recovery. If the probe fails, it returns to `OPEN` with exponential back-off multiplying the reset timeout after each failed cycle.

Before sending any request, the handler checks the breaker state:

```typescript
// src/sse/handlers/chat.ts
if (!breaker.canExecute()) {
  // circuit is OPEN → skip this provider, fall back to next one
  continue;
}

```

## Emergency Fallback Mechanism

When all providers in the fallback chain are exhausted, OmniRoute attempts a single **emergency fallback** to a free, low-cost provider defined in [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts).

### Single-Attempt Safety Guard

The emergency mechanism uses the runtime flag `emergencyFallbackTried` to prevent infinite loops:

```typescript
// src/sse/handlers/chat.ts
if (!runtimeOptions.emergencyFallbackTried && !comboName) {
  const emergencyResult = await tryEmergencyFallback(...);
  runtimeOptions.emergencyFallbackTried = true;
}

```

This guard ensures the emergency provider is attempted only once per request and only when no combo fallback remains applicable.

### Output Protection

The emergency fallback caps response length to prevent runaway output from free-tier providers, protecting against unexpected costs or resource exhaustion when operating in degraded mode.

## Request Flow Integration

The complete provider fallback mechanism operates through a coordinated pipeline in the chat request handler.

### Pipeline Gate Evaluation

Request processing begins with `checkPipelineGates`, which evaluates connection constraints and the provider's circuit breaker:

```typescript
// src/sse/handlers/chat.ts
const pipelineGates = checkPipelineGates({
  connection: activeConnection,
  breaker: getCircuitBreaker(provider)
});

```

If the gate rejects due to an `OPEN` circuit, the system records the rejection via `recordRejectedRequestUsage` and immediately consults the fallback chain.

### Complete Execution Path

The full resolution flow follows these steps:

1. **Route resolution** identifies the target provider and model
2. **Pipeline gates** evaluate the circuit breaker state
3. **Fallback loop** iterates through the chain, skipping `OPEN` circuits and recording failures
4. **Emergency fallback** triggers if the chain exhausts and `emergencyFallbackTried` is false

## Practical Implementation Examples

### Inspecting Circuit Breaker Status

Debug provider health with the circuit breaker API:

```typescript
import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";

const cb = getCircuitBreaker("openai");
console.log(cb.getStatus());
// => { name: "openai", state: "OPEN", failureCount: 7, … }

```

### Configuring Error-Specific Cooldowns

Tune resilience behavior by error type when creating breakers:

```typescript
const breaker = getCircuitBreaker("anthropic", {
  failureThreshold: 5,
  resetTimeout: 30_000,
  cooldownByKind: {
    rate_limit: 120_000,      // 2 minute back-off for throttling
    quota_exhausted: 86_400_000 // 24 hours for quota issues
  }
});

```

## Summary

- **Declarative fallback chains** in [`src/domain/fallbackPolicy.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/fallbackPolicy.ts) define priority-ordered provider alternatives using `registerFallback()` and `resolveFallbackChain()`
- **Circuit breakers** in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) implement a five-state machine (CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED) with per-error-kind cooldowns and exponential back-off
- **Emergency fallback** in [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts) provides a final safety net using the `emergencyFallbackTried` flag to prevent infinite loops
- The request handler coordinates these components through `checkPipelineGates` and the fallback loop in [`src/sse/handlers/chat.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chat.ts)

## Frequently Asked Questions

### How does OmniRoute prevent infinite loops when falling back between providers?

The system uses multiple safeguards. The `emergencyFallbackTried` runtime flag ensures the emergency provider is only attempted once per request. Additionally, the `resolveFallbackChain` function accepts an `excludedProviders` parameter that filters out providers with open circuits, preventing the loop from revisiting failed endpoints.

### What happens when a circuit breaker transitions to HALF_OPEN?

When the reset timeout expires, the breaker enters `HALF_OPEN` state and allows a single probe request through. If this request succeeds, the breaker returns to `CLOSED`. If it fails, the circuit reopens and the reset timeout multiplies exponentially, implementing progressive back-off to avoid overwhelming recovering providers.

### Can specific error types bypass the circuit breaker?

Yes. The `isFailure` callback passed to `getCircuitBreaker()` controls which errors count toward the threshold. By default, OmniRoute excludes local stream lifecycle errors using `!isLocalStreamLifecycleError(e)`, ensuring transient client disconnections don't falsely penalize provider health scores.

### Where is the fallback chain stored and how is it accessed at runtime?

Fallback chains are persisted in SQLite within the `domainState` table and cached in an in-memory `Map` for fast lookup. The [`src/domain/policyEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/policyEngine.ts) module calls `resolveFallbackChain()` to retrieve the current chain, which returns entries sorted by priority with disabled providers filtered out.