How OmniRoute’s Circuit Breaker Pattern Prevents Cascade Failures Across Combo Targets

OmniRoute isolates failing providers from its combo routing engine by consulting a per-provider circuit breaker before every target attempt, immediately skipping any provider in the OPEN state to eliminate retry cascades and resource exhaustion.

The circuit breaker pattern in OmniRoute acts as a critical safeguard within the open-sse/services/combo.ts routing engine. By evaluating provider health through individual breakers maintained in src/shared/utils/circuitBreaker.ts, the system prevents a single unreliable endpoint from triggering a cascade failure across the entire combo target chain.

The Four-State Circuit Breaker Machine

Each provider maintains an independent state machine with four distinct states defined in src/shared/utils/circuitBreaker.ts lines 5-7.

CLOSED (Normal Operation)

When the breaker is CLOSED, all requests flow through normally. This is the default healthy state where the provider responds successfully within configured thresholds.

DEGRADED (Failure Counting)

In the DEGRADED state, the breaker tracks consecutive failures via recordFailure() while still permitting traffic. This acts as an early warning buffer before the circuit opens completely.

OPEN (Hard Short-Circuit)

Once failure thresholds breach the limit, the breaker transitions to OPEN. According to lines 255-263 of src/shared/utils/circuitBreaker.ts, this state immediately returns a CircuitBreakerOpenError for every request. The canExecute() method (lines 298-302) returns false, causing the combo engine to bypass the provider entirely without attempting network connections.

HALF_OPEN (Probing for Recovery)

After the circuitBreakerReset timeout expires, the breaker enters HALF_OPEN (lines 269-277). During this phase, a limited number of halfOpenAllowed probe requests pass through to test provider health. Successful probes restore the CLOSED state; failures re-open the circuit and increment the open-cycle counter (lines 449-459).

Integration with the Combo Routing Engine

The combo engine in open-sse/services/combo.ts implements the protection at the core of its target resolution logic. Before attempting any provider in the ordered target list, the engine calls getCircuitBreaker(provider).canExecute() around lines 1720-1730.

const cb = getCircuitBreaker(provider);
if (!cb.canExecute()) {
  // skip this target, move to the next one
  continue;
}

When canExecute() returns false, the combo does not waste latency or resources on that provider, and it immediately falls back to the next target in the list. This prevents the "cascade" where one failing provider would cause retries, time-outs, and eventual exhaustion of all combo attempts.

Failure Detection and Threshold Logic

When a provider returns an error matching the configured failure-kind (such as HTTP 502 or provider-specific circuit-open messages), the combo engine invokes recordProviderFailure(). This delegates to recordFailure() in lines 408-422 of the circuit breaker utility.

If the accumulated failure count exceeds circuitBreakerThreshold, the breaker jumps directly to OPEN (lines 422-426), bypassing the DEGRADED intermediate state for severe errors. This aggressive short-circuiting prevents wasted latency on known-bad providers.

Automatic Recovery and Probing

Recovery happens automatically without manual intervention. After the reset timeout, the HALF_OPEN state allows controlled traffic through. As implemented in lines 449-459, the breaker monitors probe results to determine whether to restore full service or re-enter the OPEN state, preventing the "hammer-again" scenario when unstable providers flicker between healthy and failing.

Operational Monitoring and Manual Control

Operators gain visibility through src/lib/monitoring/providerHealthMatrix.ts and manual control via the admin API at src/app/api/resilience/reset/route.ts. Posting to this endpoint with a provider name calls getCircuitBreaker(name).reset(), immediately restoring a stuck breaker to CLOSED (lines 14-21).

# Reset a specific provider

curl -X POST https://omniroute.example.com/api/v1/resilience/reset \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"name":"openai"}'

Summary

• Per-provider isolation – Each target maintains an independent circuit breaker that prevents individual failures from propagating across the combo chain.
• Four-state protection – The state machine (CLOSED, DEGRADED, OPEN, HALF_OPEN) with automatic recovery probes ensures providers only receive traffic when demonstrably healthy.
• Pre-flight checks – The combo engine queries canExecute() before every connection attempt, skipping OPEN providers entirely to avoid connection timeouts.
• Aggressive failure detection – recordFailure() transitions breakers immediately to OPEN when thresholds exceed, eliminating wasted requests on degraded endpoints.
• Operational visibility – Admin APIs and health matrices provide real-time monitoring and manual reset capabilities for production incidents.

Frequently Asked Questions

What happens when a circuit breaker is OPEN in OmniRoute?

When a breaker is OPEN, the canExecute() method returns false and throws CircuitBreakerOpenError. The combo engine catches this condition before initiating any network connection, skipping the provider and moving immediately to the next target in the sequence. This prevents connection timeouts and resource exhaustion on known-failing endpoints.

How does OmniRoute detect when to open a circuit?

The system calls recordFailure() for errors matching configured failure kinds (such as HTTP 502 responses or provider-specific error messages). When the consecutive failure count exceeds the circuitBreakerThreshold parameter, the state machine transitions directly to OPEN, bypassing the DEGRADED intermediate state for severe errors as shown in lines 422-426 of circuitBreaker.ts.

Can operators manually reset a circuit breaker?

Yes. The /app/api/resilience/reset/route.ts endpoint accepts POST requests containing a provider name. The handler retrieves the specific breaker instance via getCircuitBreaker(name) and invokes reset(), instantly restoring the provider to the CLOSED state regardless of current failure counts or timeouts.

How does the HALF_OPEN state prevent premature traffic restoration?

HALF_OPEN allows only a limited number of probe requests—controlled by the halfOpenAllowed configuration—to test provider recovery. If these probes fail, the breaker immediately re-opens and increments the open-cycle counter (lines 449-459). This ensures unstable providers that flicker between healthy and failing states cannot receive full traffic loads until they demonstrate sustained reliability.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}