How the Circuit Breaker Pattern is Implemented in OmniRoute for Provider Request Resilience

OmniRoute implements a production-grade circuit breaker in src/shared/utils/circuitBreaker.ts that tracks five distinct states, persists state to SQLite, and uses granular failure thresholds to isolate failing provider endpoints before they exhaust system resources.

The circuit breaker pattern in OmniRoute prevents cascading failures across AI provider endpoints by automatically detecting unhealthy targets and temporarily blocking traffic to them. This implementation goes beyond basic open/closed states to include a degraded tier, per-error-type thresholds, and exponential back-off strategies. All circuit breaker instances are persisted to the database and managed through a size-limited registry to ensure resilience survives process restarts.

Core State Machine and Transitions

The circuit breaker maintains five discrete states defined in src/shared/utils/circuitBreaker.ts (lines 12‑17):

• CLOSED: Normal operation; requests pass through
• DEGRADED: Elevated failure rate detected but traffic still allowed
• OPEN: Circuit is tripped; requests fail fast immediately
• HALF_OPEN: Probing state to test if the provider has recovered
• (Implicit fifth state transition logic)

State transitions occur based on configurable failure counts and timeout timers. When the breaker enters the OPEN state, it starts a reset timer before attempting to transition to HALF_OPEN.

Request Execution and Failure Handling

Every provider request flows through the CircuitBreaker.execute() method. The execution logic (lines 34‑51) follows this flow:

1. Check if the breaker is OPEN or HALF_OPEN with no remaining probe slots
2. If blocked, throw a CircuitBreakerOpenError immediately without calling the provider
3. If allowed, execute the wrapped function
4. On success: call _onSuccess() (lines 57‑75) to potentially reset failure counters or close the circuit
5. On failure: call _onFailure() (lines 59‑75) to increment counters and potentially trigger a state change

import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';
import { fetchProvider } from '@/open-sse/executors/default';

const breaker = getCircuitBreaker('openai:gpt-4', {
  failureThreshold: 5,
  resetTimeout: 30_000,
  halfOpenRequests: 1,
});

// This will throw CircuitBreakerOpenError if the circuit is open
await breaker.execute(() => fetchProvider(...));

Granular Failure Thresholds and Adaptive Back-Off

Unlike simple counters, OmniRoute's circuit breaker can distinguish between failure kinds (rate limits, quota exhaustion, transient errors) via the kindThresholds configuration (lines 58‑74). Each error category supports:

• Distinct thresholds: Different failure counts required to trip the circuit
• Custom cooldowns: Per-error-type timeout durations
• immediateOpen flag: Skip the DEGRADED phase and jump directly to OPEN for critical errors

The breaker implements adaptive exponential back-off (lines 22‑32). After each OPEN → HALF_OPEN → OPEN cycle, the reset timeout multiplies by a configurable factor up to maxBackoffMultiplier, preventing aggressive retry storms against permanently failing endpoints.

Degradation Tier for Graceful Degradation

Before completely opening the circuit, the breaker enters a DEGRADED state (lines 83‑86, 92‑99). This occurs when failures exceed 60% of the configured failureThreshold (default behavior). In this state:

• Traffic continues to flow to the provider
• The system logs warnings for observability
• Users experience degraded service rather than complete outages

This tier provides a buffer zone where intermittent issues can resolve without triggering a full circuit break.

Persistence and Registry Management

Circuit breaker state survives process restarts through SQLite persistence (lines 62‑73, 98‑106). The _persistToDb() method saves the current state, counters, and configuration options to the domainState table, while _restoreFromDb() rehydrates the breaker on initialization. This ensures that failure history is not lost during deployments or crashes.

The system maintains a global Map registry with a hard limit of 500 active breakers (MAX_REGISTRY_SIZE). An interval sweeps the registry every 30 minutes to evict:

• Idle breakers in the CLOSED state
• The oldest entries when the registry exceeds its size limit

This prevents memory leaks in long-running processes with dynamic provider configurations.

Integration in the Provider Request Pipeline

The circuit breaker integrates directly into the request routing layer. In open-sse/services/combo.ts, the combo routing engine obtains a breaker instance via getCircuitBreaker(name, options) (lines 33‑85) and validates execution permission through breaker.canExecute() before dispatching requests.

For observability, the getAllCircuitBreakerStatuses() function (lines 86‑99) exposes the current state of all registered breakers, enabling real-time monitoring dashboards and alerting systems.

const breaker = getCircuitBreaker('anthropic:claude', {
  onStateChange: (name, oldState, newState) => {
    logger.info(`Breaker ${name} moved ${oldState} → ${newState}`);
  },
});

Summary

• Five-state machine: Tracks CLOSED, DEGRADED, OPEN, HALF_OPEN, and transition states with configurable thresholds
• Fail-fast protection: CircuitBreaker.execute() throws CircuitBreakerOpenError immediately when the circuit is open, preventing resource exhaustion
• Granular failure handling: kindThresholds allows different error types (rate limits, quotas) to trigger distinct circuit behaviors
• Adaptive back-off: Reset timeout multiplies exponentially after each failed recovery attempt up to maxBackoffMultiplier
• Graceful degradation: The DEGRADED state (triggered at 60% of failure threshold) maintains service while warning operators
• SQLite persistence: State survives restarts via _persistToDb() and _restoreFromDb() calls to the domainState table
• Registry limits: Hard cap of 500 breakers with automatic eviction of idle entries after 30 minutes
• Pipeline integration: getCircuitBreaker() and canExecute() are called in open-sse/services/combo.ts before every provider request

Frequently Asked Questions

How does OmniRoute's circuit breaker differ from a simple open/closed implementation?

OmniRoute extends the standard pattern with a DEGRADED state that allows traffic at 60% of the failure threshold, per-error-type thresholds via kindThresholds, and SQLite persistence that maintains state across restarts. According to the source code in src/shared/utils/circuitBreaker.ts, it also implements exponential back-off multipliers rather than fixed retry intervals.

What happens when a circuit breaker is OPEN in OmniRoute?

When the circuit is OPEN, the execute() method immediately throws a CircuitBreakerOpenError without attempting the underlying request (lines 34‑51). This fail-fast behavior prevents resource exhaustion and gives the provider time to recover while the breaker waits for the resetTimeout before transitioning to HALF_OPEN to test recovery.

How does the circuit breaker maintain state across application restarts?

The implementation calls _persistToDb() on every state transition, saving the current state, counters, and options to the SQLite-backed domainState table (lines 62‑73, 98‑106). On initialization, _restoreFromDb() rehydrates the breaker from this persistent storage, ensuring failure history survives deployments and crashes.

Can different types of provider errors trigger the circuit breaker differently?

Yes. The kindThresholds configuration (lines 58‑74) allows distinct thresholds, cooldowns, and the immediateOpen flag for specific error categories like rate limits or quota exhaustion. This means a transient network error might require 5 failures to open the circuit, while an authentication error could open it immediately.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}