How OmniRoute's Provider Circuit Breaker Prevents Cascading Failures

OmniRoute prevents cascading failures by isolating errors at the provider level through independent circuit breakers that automatically block traffic to failing providers and route requests to healthy alternatives.

Each provider in OmniRoute—whether OAuth-based (OpenAI, Anthropic), API-key-based (Google), or local—runs its own circuit breaker that tracks transient HTTP errors. When failures exceed type-specific thresholds, the breaker opens to protect the system and enables graceful failover to other providers.

Understanding the Circuit Breaker States

OmniRoute implements a three-state circuit breaker pattern in src/shared/utils/circuitBreaker.ts. Each provider maintains independent state, ensuring that problems with one upstream service cannot propagate across the routing layer.

CLOSED State (Normal Operation)

In the CLOSED state, the circuit breaker permits all traffic to the provider. The system continuously monitors responses for error codes:

• 408 (Request Timeout)
• 500 (Internal Server Error)
• 502 (Bad Gateway)
• 503 (Service Unavailable)
• 504 (Gateway Timeout)

These transient failure codes increment an internal counter. When the counter reaches the threshold for that provider type, the breaker transitions to OPEN.

OPEN State (Failure Isolation)

Once OPEN, the circuit breaker immediately blocks all new traffic to the affected provider. The routing layer in open-sse/handlers/chatHelpers.ts queries breaker status before making requests, automatically skipping providers that are not CLOSED.

The default thresholds vary by provider type as defined in open-sse/config/constants.ts:

Provider Type	Failure Threshold	Reset Timeout
OAuth providers	3 errors	60 seconds
API-key providers	5 errors	30 seconds
Local providers	2 errors	15 seconds

HALF-OPEN State (Recovery Probe)

After the reset timeout expires, the breaker does not immediately restore service. Instead, it enters HALF-OPEN on the next status check, allowing a single probe request:

• Success: Returns to CLOSED, resuming normal traffic
• Failure: Re-opens the circuit, resetting the timeout

This lazy transition prevents premature restoration to a still-failing service.

Provider-Level Failure Isolation in Practice

The combinatorial routing logic leverages breaker status to implement intelligent failover. When a request arrives, the system evaluates provider availability through getProviderStatus() before making allocation decisions.

Checking Provider Status Before Routing

import { getProviderStatus } from '@/shared/utils/circuitBreaker';

// Inside the routing logic
if (getProviderStatus('openai') === 'CLOSED') {
  // Route request to OpenAI
} else {
  // Fallback to another provider
}

This pattern appears throughout open-sse/handlers/chatHelpers.ts, where the routing pipeline filters the provider pool to only those with CLOSED breakers.

Account-Level Fallback Logic

The open-sse/services/accountFallback.ts module extends this protection to individual account credentials. When a specific account encounters repeated failures, the breaker status propagates to exclude that account from rotation while preserving other accounts for the same provider.

Configuring Circuit Breaker Thresholds

All timeout and threshold values are centralized in open-sse/config/constants.ts. These defaults balance fast failure detection against avoiding over-sensitive tripping:

• OAuth providers (3 errors, 60s timeout): Higher tolerance for intermittent issues due to token refresh complexity
• API-key providers (5 errors, 30s timeout): Moderate threshold for generally stable services
• Local providers (2 errors, 15s timeout): Aggressive detection for infrastructure that should rarely fail

Operators can adjust these constants and redeploy without modifying core logic.

Administrative Circuit Breaker Control

For operational scenarios requiring intervention, OmniRoute exposes resetProviderBreaker() to force-clear a stuck or misconfigured breaker state.

Manually Resetting a Provider Breaker

import { resetProviderBreaker } from '@/shared/utils/circuitBreaker';

// Admin command
resetProviderBreaker('anthropic');

This bypasses the normal timeout and state machine, immediately returning the provider to CLOSED. Use cautiously—manual resets risk overwhelming a genuinely struggling upstream service.

How This Prevents Cascading Failures

Without circuit breakers, a degraded provider would cause:

1. Retry storms as clients repeatedly attempt failed requests
2. Resource exhaustion from pending connections and timeouts
3. Latency propagation as the routing layer stalls waiting for responses
4. Cross-provider impact if the proxy itself becomes overloaded

OmniRoute's design contains each failure:

• Spatial isolation: Each provider runs an independent breaker
• Temporal protection: Open breakers reject requests instantly without backend calls
• Automatic recovery: HALF-OPEN probes restore service only when healthy
• Graceful degradation: Traffic shifts to alternative providers transparently

The result is a resilient multi-provider proxy where temporary upstream outages remain localized rather than systemic.

Summary

• Independent breakers per provider in src/shared/utils/circuitBreaker.ts isolate failures to specific upstream services
• Three-state machine (CLOSED → OPEN → HALF-OPEN) enables automatic detection, blocking, and recovery
• Type-specific thresholds in open-sse/config/constants.ts tune sensitivity for OAuth, API-key, and local providers
• Integration with routing layer ensures failed providers are skipped without manual intervention
• Administrative reset capability provides escape hatch for operational edge cases

Frequently Asked Questions

What HTTP status codes trigger the OmniRoute circuit breaker?

OmniRoute monitors for transient server-side errors: 408, 500, 502, 503, and 504. These indicate temporary problems rather than permanent failures like authentication errors (401, 403) or client errors (400, 404), which do not increment the breaker counter.

How long does a provider stay blocked when the circuit breaker opens?

The reset timeout depends on provider type: 60 seconds for OAuth providers, 30 seconds for API-key providers, and 15 seconds for local providers. The breaker only transitions to HALF-OPEN on the next status query after this timeout expires—not automatically—ensuring at least the full timeout period of protection.

Can I manually override a circuit breaker if I know the provider is healthy?

Yes. Import resetProviderBreaker from src/shared/utils/circuitBreaker.ts and call it with the provider identifier (e.g., resetProviderBreaker('openai')). This immediately returns the breaker to CLOSED regardless of state or timeout remaining.

What happens to requests when all providers have open circuit breakers?

The routing logic in open-sse/handlers/accountFallback.ts handles this edge case by attempting to select the provider with the least severe failure history or returning a service unavailable response if no viable provider exists. This prevents infinite loops or undefined behavior when the entire upstream ecosystem is degraded.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}