How OmniRoute's Circuit Breaker Handles Provider Failures: State Machine & Recovery Strategy

OmniRoute uses a persistent three-state circuit breaker in src/shared/utils/circuitBreaker.ts that tracks upstream failures per provider, opens the circuit when thresholds are exceeded, and lazily transitions to half-open after timeouts to probe for recovery.

The diegosouzapw/OmniRoute repository implements a resilient routing layer that shields the platform from unreliable upstream AI providers. The circuit breaker monitors HTTP error patterns, maintains durable state across process restarts, and integrates directly with the provider selection logic to ensure only healthy endpoints receive traffic.

Three-State Circuit Breaker Architecture

The core implementation follows the classic circuit breaker pattern with three distinct states defined in src/shared/utils/circuitBreaker.ts:

• CLOSED: Traffic flows normally to the provider. The breaker counts consecutive failures in this state.
• OPEN: All requests to the provider are blocked immediately. The breaker remains open until the configured reset timeout expires.
• HALF_OPEN: A probationary state allowing a single probe request to test if the provider has recovered.

State transitions occur automatically through lazy evaluation. When code calls canExecute() or getStatus(), the breaker checks the elapsed time since entering OPEN. If the timeout has expired, it atomically transitions to HALF_OPEN for the next request. A successful execution in HALF_OPEN resets the breaker to CLOSED, while a failure reverts it immediately to OPEN with a fresh timeout.

Failure Detection and Classification

Not all errors count as circuit breaker failures. In open-sse/handlers/chatCore.ts, the isLocalStreamLifecycleError function classifies errors to distinguish between transient upstream issues and authentication failures.

The circuit breaker increments its failure counter only for upstream service errors including HTTP 408 (Request Timeout), 500 (Internal Server Error), 502 (Bad Gateway), 503 (Service Unavailable), and 504 (Gateway Timeout). These are recorded via recordFailure('upstream').

Authentication-related errors such as 401 (Unauthorized), 403 (Forbidden), and 429 (Rate Limited) are intentionally excluded from the circuit breaker logic. These trigger separate mechanisms like connection cooldowns or model lockouts managed by open-sse/services/accountFallback.ts, preventing the breaker from opening due to credential issues or quota violations.

Provider-Specific Thresholds and Configuration

Thresholds and reset timeouts vary by provider authentication type and are defined in src/shared/constants/providers.ts:

• OAuth providers: Threshold of 3 failures, reset timeout of 60 seconds
• API-key providers: Threshold of 5 failures, reset timeout of 30 seconds
• Local (self-hosted) providers: Threshold of 2 failures, reset timeout of 15 seconds

This differentiation recognizes that self-hosted infrastructure typically requires faster failure detection, while managed OAuth endpoints may tolerate brief transient issues.

Persistence and State Management

The circuit breaker maintains durable state in the domain_circuit_breakers database table, ensuring failure counts and state transitions survive process restarts. The public API exposes three primary functions in src/shared/utils/circuitBreaker.ts:

• getCircuitBreaker(name): Retrieves or initializes a breaker instance for a specific provider
• canExecute(): Returns boolean indicating whether the provider is currently reachable (CLOSED or HALF_OPEN)
• getStatus(): Returns the current state object including failure counts and timestamps

After a successful request, callers must invoke recordSuccess() to reset the failure accumulator to zero. Conversely, recordFailure(kind) increments counters and triggers the OPEN transition when thresholds are breached.

Integration with the Routing Layer

The combo router (open-sse/services/combo/targetSorters.ts and quotaStrategies.ts) consults the circuit breaker during provider selection. Before adding a provider to the candidate pool, the router checks getCircuitBreaker(provider).getStatus(). Providers in the OPEN state are automatically excluded from routing decisions.

The resilience explanation system (src/lib/usage/resilienceExplain.ts) also consumes breaker status to generate health reports, allowing dashboards to display why specific providers are temporarily unavailable.

Token Refresh Isolation

Token refresh flows maintain a separate lightweight circuit breaker in open-sse/services/tokenRefresh/circuitBreaker.ts. This isolation prevents OAuth refresh failures from impacting the main provider circuit while still protecting the token endpoint from excessive retry traffic during outages.

Summary

• Three-state machine: CLOSED tracks failures, OPEN blocks traffic, HALF_OPEN probes for recovery via lazy state transitions in src/shared/utils/circuitBreaker.ts
• Selective failure counting: Only upstream HTTP errors (408, 500, 502, 503, 504) trigger the breaker; auth errors (401, 403, 429) are handled separately in open-sse/handlers/chatCore.ts
• Configurable thresholds: Provider types in src/shared/constants/providers.ts define specific failure thresholds (2-5) and reset timeouts (15-60 seconds)
• Durable state: Breaker status persists to domain_circuit_breakers table, with getCircuitBreaker(), canExecute(), and recordSuccess() forming the primary API
• Routing integration: The combo router filters OPEN providers via targetSorters.ts, while a separate breaker in tokenRefresh/circuitBreaker.ts handles OAuth refresh isolation

Frequently Asked Questions

What triggers the circuit breaker to open in OmniRoute?

The circuit breaker transitions from CLOSED to OPEN when recordFailure() is called for an upstream error (HTTP 408, 500, 502, 503, or 504) and the consecutive failure count reaches the provider-specific threshold defined in src/shared/constants/providers.ts. Authentication errors like 401 or 429 do not increment the failure counter.

How does the circuit breaker recover from an OPEN state?

Recovery uses lazy state transition. When canExecute() or getStatus() is called after the reset timeout expires, the breaker automatically transitions to HALF_OPEN, allowing a single probe request. If that request succeeds and recordSuccess() is called, the breaker resets to CLOSED. If it fails, the breaker returns to OPEN with a fresh timeout.

Are authentication errors like 401 or 403 counted by the circuit breaker?

No. The isLocalStreamLifecycleError classifier in open-sse/handlers/chatCore.ts explicitly excludes 401, 403, and 429 errors from circuit breaker consideration. These trigger connection cooldown mechanisms instead, preventing the breaker from opening due to credential or quota issues rather than service unavailability.

Where is the circuit breaker state stored in OmniRoute?

Provider circuit breaker state persists in the domain_circuit_breakers database table. The getCircuitBreaker(name) function in src/shared/utils/circuitBreaker.ts handles read/write operations, ensuring that failure counts and OPEN/HALF_OPEN states survive application restarts or horizontal scaling events.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}