How the Circuit Breaker in OmniRoute Prevents Cascading Failures

OmniRoute's circuit breaker monitors provider health and automatically blocks requests to failing services after a threshold of errors, stopping cascading failures before they propagate through the routing engine.

The circuit breaker in OmniRoute is a critical resilience mechanism implemented in the diegosouzapw/OmniRoute repository that protects the combo-routing engine from repeatedly invoking unhealthy providers. By detecting failure patterns—such as HTTP 429 rate limits, network timeouts, or connection errors—it isolates problematic downstream services to maintain system stability and prevent resource exhaustion across the platform.

Understanding the Three-State Circuit Breaker Pattern

OmniRoute implements the classic circuit breaker pattern with three distinct states that automatically manage traffic flow to failing providers:

• CLOSED: Requests pass through normally to the provider. The breaker tracks failure counts incrementally with each error response.
• OPEN: After exceeding the configured failure threshold, the breaker blocks all requests to the provider for a configurable cool-down period. Calls immediately fail with a CircuitBreakerOpenError without reaching the downstream service.
• HALF-OPEN: Once the cool-down expires, the breaker allows a limited number of trial requests to test if the provider has recovered. Successful trials reset the state to CLOSED, while failures return it to OPEN.

This state machine lives in src/shared/utils/circuitBreaker.ts and operates per-provider, ensuring that a single failing endpoint cannot overwhelm the routing system.

Core Implementation Details

The circuit breaker implementation centers on the CircuitBreaker class exported from src/shared/utils/circuitBreaker.ts. This module provides the primary interface for creating, executing, and monitoring breaker instances across the application.

Provider-Specific Configuration

Provider profiles in OmniRoute expose two key configuration parameters to fine-tune failure detection:

• circuitBreakerThreshold: The number of consecutive failures required to transition from CLOSED to OPEN.
• circuitBreakerReset: The duration (in milliseconds) the breaker remains OPEN before transitioning to HALF-OPEN.

These values are validated in the test suite (see tests/unit/error-classification.test.ts) to ensure each provider maintains appropriate sensitivity levels based on their reliability characteristics and rate-limit policies.

Integration with the Combo Routing Engine

The combo routing engine leverages circuit breaker state to make intelligent routing decisions. Before dispatching any request, the engine queries the provider's current circuitBreakerState (as implemented in src/open-sse/services/combo.ts). If the breaker is OPEN, the router automatically falls back to alternative providers or lower-priority combos, ensuring continuous service availability even when specific integrations fail.

Practical Usage Examples

Initializing a Circuit Breaker for a Provider

Use the getCircuitBreaker factory function to retrieve or lazily create a breaker instance for a specific provider. You can optionally attach custom error classification logic to distinguish between transient failures and critical errors.

import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';
import { HttpError } from '@/shared/errors';

// Obtain (or lazily create) a breaker for the "openai" provider
const openaiBreaker = getCircuitBreaker('openai');

// Configure a custom failure classifier (optional)
openaiBreaker.classifyError = (err) => {
  // Treat HTTP 429 as a rate-limit failure, everything else as generic
  return err instanceof HttpError && err.status === 429 ? 'RATE_LIMIT' : undefined;
};

Executing Requests Through the Breaker

Wrap provider calls using the execute method, which only runs the supplied async function if the breaker is not OPEN. Otherwise, it throws CircuitBreakerOpenError immediately.

// breaker.execute runs the supplied async function only if the breaker
// is not OPEN; otherwise it throws CircuitBreakerOpenError.
const response = await openaiBreaker.execute(async () => {
  return await fetchOpenAI(completionPayload);
});

Monitoring Breaker Health

For observability, the getStatus function aggregates breaker statistics across all providers, exposing counts of open, half-open, and closed circuits. This data feeds into the health-monitoring endpoint (see src/app/api/health/route.ts) to provide real-time visibility into system resilience.

import { getStatus } from '@/shared/utils/circuitBreaker';

const status = getStatus();  // → { open: 1, halfOpen: 0, closed: 2, … }
console.log('Circuit-breaker summary:', status);

Resetting Breakers During Testing

During test teardown or manual recovery scenarios, use resetAllCircuitBreakers to return every breaker instance to the CLOSED state.

import { resetAllCircuitBreakers } from '@/shared/utils/circuitBreaker';

await resetAllCircuitBreakers();   // returns every breaker to CLOSED state

Benefits for System Resilience

By short-circuiting requests to unhealthy providers, OmniRoute's circuit breaker delivers three critical protections against cascading failures:

1. Prevents Error Propagation: Failed providers cannot trigger retry storms or timeout cascades that would otherwise propagate through the combo-routing engine to the rest of the platform.
2. Conserves Resources: Blocking requests at the application layer prevents thread pool exhaustion, memory pressure, and wasted rate-limit tokens that would occur if every request attempted to reach the failing service.
3. Enables Graceful Recovery: The HALF-OPEN state allows automatic detection of service restoration without manual intervention, while fallback routing ensures users experience minimal disruption during outages.

Summary

• The circuit breaker in src/shared/utils/circuitBreaker.ts implements a three-state machine (CLOSED, OPEN, HALF-OPEN) to monitor provider health.
• Providers configure failure thresholds via circuitBreakerThreshold and recovery timeouts via circuitBreakerReset.
• The combo routing engine checks circuitBreakerState before dispatching requests, enabling automatic failover to healthy providers.
• When triggered, the breaker throws CircuitBreakerOpenError immediately, preventing resource exhaustion and cascading failures.
• Use getStatus() for observability and resetAllCircuitBreakers() for test isolation or manual recovery.

Frequently Asked Questions

What triggers a circuit breaker to open in OmniRoute?

A circuit breaker transitions to OPEN when the number of consecutive failures exceeds the provider's configured circuitBreakerThreshold. Failures include HTTP 429 responses (handled by src/shared/utils/classify429.ts), network timeouts, connection errors, or any exception classified by the breaker's classifyError function as a qualifying failure.

How does OmniRoute's circuit breaker transition from HALF-OPEN to CLOSED?

After the circuitBreakerReset cooldown period expires, the breaker enters HALF-OPEN and permits a limited number of trial requests. If these trial requests succeed, the breaker automatically resets to CLOSED, resuming normal traffic flow. If any trial fails, the breaker returns to OPEN and restarts the cooldown timer.

Can I configure different failure thresholds for different providers?

Yes. Each provider profile defines its own circuitBreakerThreshold and circuitBreakerReset values optimized for that service's reliability characteristics. High-volume providers might tolerate more failures before opening, while critical external APIs might have stricter thresholds to ensure rapid failover.

How does the combo router handle requests when a circuit breaker is open?

When circuitBreakerState returns OPEN for a primary provider, the combo routing engine (in src/open-sse/services/combo.ts) automatically excludes that provider from the routing pool and selects alternative providers or lower-priority combos. This ensures that a single flaky endpoint cannot halt the entire request flow, maintaining system availability through intelligent traffic shaping.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}