# How OmniRoute’s Circuit Breaker Pattern Prevents Cascade Failures Across Combo Targets

> Learn how OmniRoutes circuit breaker pattern stops cascade failures by isolating failing providers, preventing retries and resource exhaustion for combo targets.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-07-30

---

**OmniRoute isolates failing providers from its combo routing engine by consulting a per-provider circuit breaker before every target attempt, immediately skipping any provider in the `OPEN` state to eliminate retry cascades and resource exhaustion.**

The circuit breaker pattern in OmniRoute acts as a critical safeguard within the [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) routing engine. By evaluating provider health through individual breakers maintained in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), the system prevents a single unreliable endpoint from triggering a cascade failure across the entire combo target chain.

## The Four-State Circuit Breaker Machine

Each provider maintains an independent state machine with four distinct states defined in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) lines 5-7.

### CLOSED (Normal Operation)

When the breaker is **CLOSED**, all requests flow through normally. This is the default healthy state where the provider responds successfully within configured thresholds.

### DEGRADED (Failure Counting)

In the **DEGRADED** state, the breaker tracks consecutive failures via `recordFailure()` while still permitting traffic. This acts as an early warning buffer before the circuit opens completely.

### OPEN (Hard Short-Circuit)

Once failure thresholds breach the limit, the breaker transitions to **OPEN**. According to lines 255-263 of [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts), this state immediately returns a `CircuitBreakerOpenError` for every request. The `canExecute()` method (lines 298-302) returns `false`, causing the combo engine to bypass the provider entirely without attempting network connections.

### HALF_OPEN (Probing for Recovery)

After the `circuitBreakerReset` timeout expires, the breaker enters **HALF_OPEN** (lines 269-277). During this phase, a limited number of `halfOpenAllowed` probe requests pass through to test provider health. Successful probes restore the CLOSED state; failures re-open the circuit and increment the open-cycle counter (lines 449-459).

## Integration with the Combo Routing Engine

The combo engine in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) implements the protection at the core of its target resolution logic. Before attempting any provider in the ordered target list, the engine calls `getCircuitBreaker(provider).canExecute()` around lines 1720-1730.

```typescript
const cb = getCircuitBreaker(provider);
if (!cb.canExecute()) {
  // skip this target, move to the next one
  continue;
}

```

When `canExecute()` returns `false`, the combo **does not waste latency or resources** on that provider, and it immediately falls back to the next target in the list. This prevents the "cascade" where one failing provider would cause retries, time-outs, and eventual exhaustion of all combo attempts.

## Failure Detection and Threshold Logic

When a provider returns an error matching the configured *failure-kind* (such as HTTP 502 or provider-specific circuit-open messages), the combo engine invokes `recordProviderFailure()`. This delegates to `recordFailure()` in lines 408-422 of the circuit breaker utility.

If the accumulated failure count exceeds `circuitBreakerThreshold`, the breaker jumps directly to **OPEN** (lines 422-426), bypassing the DEGRADED intermediate state for severe errors. This aggressive short-circuiting prevents wasted latency on known-bad providers.

## Automatic Recovery and Probing

Recovery happens automatically without manual intervention. After the reset timeout, the **HALF_OPEN** state allows controlled traffic through. As implemented in lines 449-459, the breaker monitors probe results to determine whether to restore full service or re-enter the OPEN state, preventing the "hammer-again" scenario when unstable providers flicker between healthy and failing.

## Operational Monitoring and Manual Control

Operators gain visibility through [`src/lib/monitoring/providerHealthMatrix.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/monitoring/providerHealthMatrix.ts) and manual control via the admin API at [`src/app/api/resilience/reset/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/resilience/reset/route.ts). Posting to this endpoint with a provider name calls `getCircuitBreaker(name).reset()`, immediately restoring a stuck breaker to CLOSED (lines 14-21).

```bash

# Reset a specific provider

curl -X POST https://omniroute.example.com/api/v1/resilience/reset \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"name":"openai"}'

```

## Summary

- **Per-provider isolation** – Each target maintains an independent circuit breaker that prevents individual failures from propagating across the combo chain.
- **Four-state protection** – The state machine (CLOSED, DEGRADED, OPEN, HALF_OPEN) with automatic recovery probes ensures providers only receive traffic when demonstrably healthy.
- **Pre-flight checks** – The combo engine queries `canExecute()` before every connection attempt, skipping OPEN providers entirely to avoid connection timeouts.
- **Aggressive failure detection** – `recordFailure()` transitions breakers immediately to OPEN when thresholds exceed, eliminating wasted requests on degraded endpoints.
- **Operational visibility** – Admin APIs and health matrices provide real-time monitoring and manual reset capabilities for production incidents.

## Frequently Asked Questions

### What happens when a circuit breaker is OPEN in OmniRoute?

When a breaker is **OPEN**, the `canExecute()` method returns `false` and throws `CircuitBreakerOpenError`. The combo engine catches this condition before initiating any network connection, skipping the provider and moving immediately to the next target in the sequence. This prevents connection timeouts and resource exhaustion on known-failing endpoints.

### How does OmniRoute detect when to open a circuit?

The system calls `recordFailure()` for errors matching configured failure kinds (such as HTTP 502 responses or provider-specific error messages). When the consecutive failure count exceeds the `circuitBreakerThreshold` parameter, the state machine transitions directly to **OPEN**, bypassing the DEGRADED intermediate state for severe errors as shown in lines 422-426 of [`circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/circuitBreaker.ts).

### Can operators manually reset a circuit breaker?

Yes. The [`/app/api/resilience/reset/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main//app/api/resilience/reset/route.ts) endpoint accepts POST requests containing a provider name. The handler retrieves the specific breaker instance via `getCircuitBreaker(name)` and invokes `reset()`, instantly restoring the provider to the **CLOSED** state regardless of current failure counts or timeouts.

### How does the HALF_OPEN state prevent premature traffic restoration?

**HALF_OPEN** allows only a limited number of probe requests—controlled by the `halfOpenAllowed` configuration—to test provider recovery. If these probes fail, the breaker immediately re-opens and increments the open-cycle counter (lines 449-459). This ensures unstable providers that flicker between healthy and failing states cannot receive full traffic loads until they demonstrate sustained reliability.