How to Configure Circuit Breakers for Provider Resilience in OmniRoute

OmniRoute protects upstream providers from cascading failures by wrapping each routing target in a circuit breaker that automatically trips after a configurable failure threshold and resets after a cooldown period, with the core implementation located in src/shared/utils/circuitBreaker.ts.

OmniRoute is an open-source intelligent routing platform designed to handle provider instability gracefully. Configuring circuit breakers for provider resilience ensures that consecutive failures from a specific upstream service trigger an automatic circuit open, temporarily removing that provider from the routing pool until a cooldown period expires. This mechanism prevents wasted requests against unhealthy endpoints and allows the system to self-heal.

Understanding the Circuit Breaker State Machine

The circuit breaker implementation in src/shared/utils/circuitBreaker.ts maintains three distinct states for each provider: CLOSED (normal operation), OPEN (failing fast), and HALF-OPEN (testing recovery). When a provider is first accessed, the system creates a breaker instance via the getCircuitBreaker(name, options) factory function, which accepts a unique identifier (typically the provider ID) and an optional configuration object. The breaker monitors consecutive failures; once the count exceeds the configured circuitBreakerThreshold, the state transitions to OPEN and the provider is bypassed for the duration specified by circuitBreakerReset milliseconds. After this cooldown, the breaker enters HALF-OPEN and allows a single probe request to determine if the provider has recovered.

Configuring Circuit Breaker Thresholds in Provider Profiles

Provider-specific resilience parameters are defined in src/shared/constants/providers.ts within the PROVIDER_PROFILES object. Each profile can specify distinct circuitBreakerThreshold and circuitBreakerReset values based on the provider's stability characteristics.

// src/shared/constants/providers.ts
export const PROVIDER_PROFILES = {
  oauth: {
    circuitBreakerThreshold: 60,
    circuitBreakerReset: 5_000, // 5 seconds
    // additional OAuth-specific settings
  },
  apikey: {
    circuitBreakerThreshold: 30,
    circuitBreakerReset: 10_000, // 10 seconds
    // additional API key settings
  },
  openai: {
    circuitBreakerThreshold: 20,
    circuitBreakerReset: 30_000, // 30 seconds
    // model-specific configuration
  }
};

These defaults are automatically injected when calling getCircuitBreaker without explicit options, ensuring consistent resilience policies across the application.

Implementing Circuit Breaker Guards in Request Flows

To utilize circuit breaker protection in your routing logic, import the breaker utility and wrap your provider calls with state checks and outcome recording.

Initializing a Breaker Instance

Create a breaker for a specific provider using the factory function:

import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";

// Obtain or create a breaker for a specific provider
const breaker = getCircuitBreaker("openai-gpt-4", {
  threshold: 20,
  resetTimeout: 30_000
});

Guarding Requests with canExecute()

Always verify circuit state before dispatching requests to avoid hitting providers that are currently marked as unhealthy:

if (!breaker.canExecute()) {
  throw new Error("Provider circuit is OPEN; routing to fallback");
}

try {
  const response = await fetchProviderEndpoint();
  
  // Record successful response to maintain closed state
  breaker.recordSuccess();
  return response;
} catch (error) {
  // Record failure, potentially triggering state transition
  breaker.recordFailure();
  throw error;
}

The canExecute() method returns true only when the breaker is in CLOSED or HALF-OPEN states, effectively blocking traffic during the cooldown period.

Manual Reset and Administrative Control

For maintenance windows or emergency recovery scenarios, you can force a breaker back to CLOSED using the reset() method on individual instances, or clear all breakers globally:

import { resetAllCircuitBreakers } from "@/shared/utils/circuitBreaker";

// Reset a specific provider
breaker.reset();

// Reset all circuit breakers system-wide (useful in tests or admin CLI)
resetAllCircuitBreakers();

Advanced Configuration Examples

The circuit breaker pattern extends beyond basic provider routing. The file open-sse/services/tokenRefresh/circuitBreaker.ts demonstrates specialized usage for token refresh flows, where transient authentication failures should not permanently disable the service. You can override profile defaults at runtime by passing custom options to getCircuitBreaker, allowing dynamic adjustments based on load balancing requirements or real-time health metrics.

// Runtime configuration override for high-priority providers
const criticalBreaker = getCircuitBreaker("priority-api", {
  threshold: 100, // Higher tolerance for critical path
  resetTimeout: 60_000 // Longer recovery observation window
});

Summary

• Circuit breakers in OmniRoute are implemented in src/shared/utils/circuitBreaker.ts using a getCircuitBreaker(name, options) factory pattern.
• Provider profiles in src/shared/constants/providers.ts define default circuitBreakerThreshold (failure count before opening) and circuitBreakerReset (cooldown duration in milliseconds).
• Request flow requires checking canExecute() before dispatch, then calling recordSuccess() or recordFailure() based on the outcome.
• Manual intervention is available via reset() on individual breakers or resetAllCircuitBreakers() for global system reset.
• State transitions follow a CLOSED → OPEN → HALF-OPEN → CLOSED lifecycle that automatically isolates failing providers and tests for recovery.

Frequently Asked Questions

How does OmniRoute determine when to open a circuit breaker?

OmniRoute increments an internal failure counter each time recordFailure() is called on a breaker instance. According to the implementation in src/shared/utils/circuitBreaker.ts, when this counter exceeds the circuitBreakerThreshold defined in the provider profile or passed as an option to getCircuitBreaker(), the state immediately transitions to OPEN, preventing further requests to that provider until the cooldown completes.

Can I configure different thresholds for different providers?

Yes. The PROVIDER_PROFILES exported from src/shared/constants/providers.ts allows you to define provider-specific resilience characteristics. Pass unique circuitBreakerThreshold and circuitBreakerReset values for each provider type (such as oauth versus apikey), and the system will instantiate breakers with those parameters when getCircuitBreaker is called with the corresponding provider name.

What happens to requests when a circuit breaker is open?

When a breaker is in the OPEN state, the canExecute() method returns false, causing the routing layer to skip that provider and either route to the next available candidate in the pool or return an error indicating no healthy providers are available. This "fail-fast" behavior prevents network congestion and resource exhaustion from repeated attempts against known unhealthy endpoints.

How do I manually reset a circuit breaker in OmniRoute?

You have two options for manual intervention. Call breaker.reset() on a specific circuit breaker instance to immediately return it to the CLOSED state, or import and invoke resetAllCircuitBreakers() from @/shared/utils/circuitBreaker to clear all breaker states system-wide. These functions are particularly useful during testing scenarios or when you need to force a provider back into rotation after deploying a fix.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}