How OmniRoute's Circuit Breaker Handles Provider Failures Across Connection Types

OmniRoute uses a single, reusable circuit breaker implementation in src/shared/utils/circuitBreaker.ts to protect all outbound provider connections—HTTP/REST, SSE streams, OAuth token refresh, and TLS tunnels—distinguishing failure kinds and applying per-kind thresholds, cool-down overrides, and shared state transitions.

OmniRoute's circuit breaker architecture centralizes failure management for diverse connection types. Whether you're calling OpenAI's REST API, streaming responses via Server-Sent Events, refreshing OAuth tokens, or establishing TLS tunnels, the same core state machine governs provider health—while allowing connection-specific customization through configuration options.

Core Circuit Breaker Implementation

The circuit breaker implementation resides in src/shared/utils/circuitBreaker.ts and exports the getCircuitBreaker factory function. All connection types ultimately delegate to this single source of truth.

State Machine and Transitions

The breaker implements a five-state lifecycle:

CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED

• CLOSED: Normal operation; requests pass through
• DEGRADED: Warning state after degradationThreshold failures; requests still proceed
• OPEN: Requests blocked immediately; CircuitBreakerOpenError thrown
• HALF_OPEN: Single probe request allowed to test provider recovery

State constants are defined at lines 12-17 in src/shared/utils/circuitBreaker.ts.

Failure Kind Awareness

The classifyError callback transforms raw errors into typed FailureKind values. The CircuitBreakerOptions.kindThresholds (lines 96-100) allows each kind to specify:

• threshold: Count before state transition
• cooldown: Override for global reset timeout
• immediateOpen: Skip DEGRADED and jump straight to OPEN

type FailureKind = "rate_limit" | "quota_exhausted" | "transient" | "custom";

// From CircuitBreakerOptions interface
kindThresholds?: Partial<Record<FailureKind, {
  threshold: number;
  cooldown?: number;
  immediateOpen?: boolean;
}>>;

Adaptive Back-off and Per-Kind Cool-downs

After each OPEN → HALF_OPEN → OPEN cycle, the reset timeout grows exponentially. The _effectiveResetTimeout implementation (lines 41-49) caps growth via maxBackoffMultiplier to prevent indefinite waiting.

The _effectiveCooldown logic (lines 31-38) checks for cooldownByKind entries first, allowing "rate_limit" errors to recover faster than generic transient failures.

Persistence Across Restarts

State durability is handled through _persistToDb and _restoreFromDb (lines 17-25). The breaker serializes its state to the domain database, ensuring multi-process deployments share consistent health views across restarts.

HTTP/REST Provider Integration

REST providers like OpenAI and Claude instantiate breakers in src/lib/providers/* modules. The classifyError function maps HTTP status codes to failure kinds.

import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";

const openAiBreaker = getCircuitBreaker("openai", {
  failureThreshold: 5,
  resetTimeout: 30_000,
  classifyError: (err) => {
    if (err?.status === 429) return "rate_limit";
    if (err?.status === 403) return "quota_exhausted";
    return "transient";
  },
});

Execution wraps the fetch call:

async function callOpenAI(payload: any) {
  return openAiBreaker.execute(async () => {
    const resp = await fetch("https://api.openai.com/v1/chat/completions", {
      method: "POST",
      headers: { Authorization: `Bearer ${process.env.OPENAI_KEY}` },
      body: JSON.stringify(payload),
    });
    if (!resp.ok) throw { status: resp.status, body: await resp.text() };
    return resp.json();
  });
}

Three consecutive 429 responses transition the breaker to OPEN, blocking subsequent requests until the 30-second cooldown expires.

SSE Streaming Protection

In open-sse/handlers/chat.ts, streaming endpoints use the same breaker pattern with one key addition: the isFailure predicate filters local stream lifecycle errors that shouldn't count against the provider.

import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";
import { streamChat } from "@/open-sse/handlers/chat";

const chatBreaker = getCircuitBreaker("codex", {
  failureThreshold: 4,
  resetTimeout: 20_000,
  classifyError,
  isFailure: (e) => !isLocalStreamLifecycleError(e),
});

export async function chatEndpoint(req) {
  return chatBreaker.execute(() => streamChat(req));
}

This prevents client disconnections or proxy hiccups from falsely degrading provider health.

OAuth Token Refresh Circuit Breaking

Token refresh flows in open-sse/services/tokenRefresh/circuitBreaker.ts wrap the core breaker with OAuth-optimized defaults:

import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";

export const tokenRefreshBreaker = getCircuitBreaker("tokenRefresh:google", {
  failureThreshold: 5,
  resetTimeout: 5_000,  // Fast recovery for temporary OAuth hiccups
});

The short 5-second reset timeout prevents aggressive retry loops that could trigger OAuth rate limits, while still allowing rapid recovery from transient identity provider issues.

TLS Tunnel Isolation

TLS client connections in open-sse/services/tlsClient/circuitBreaker.ts use immediateOpen: true for handshake failures:

const tlsBreaker = getCircuitBreaker(`tls:${endpoint}`, {
  failureThreshold: 3,
  resetTimeout: 60_000,
  kindThresholds: {
    tls_handshake_failed: {
      threshold: 1,
      immediateOpen: true,  // Isolate broken tunnels instantly
    },
  },
});

A failed TLS handshake indicates fundamental connectivity problems—certificate mismatches, MITM interference, or endpoint compromise—requiring immediate circuit isolation rather than gradual degradation.

A2A Webhook Protection

Agent-to-agent webhooks in src/lib/a2a/* inherit global defaults but can supply custom kindThresholds for high-traffic endpoints:

const webhookBreaker = getCircuitBreaker("a2a:partner-erp", {
  kindThresholds: {
    quota_exhausted: {
      threshold: 2,       // Faster open for quota issues
      cooldown: 300_000,  // 5-minute recovery for partner limits
    },
  },
});

Summary

• Single implementation: All connection types use src/shared/utils/circuitBreaker.ts via getCircuitBreaker()
• Failure kind discrimination: HTTP 429s, 403s, TLS errors, and OAuth failures each get tailored thresholds and cooldowns
• Adaptive behavior: Exponential back-off prevents hammering; per-kind overrides fine-tune recovery timing
• State persistence: Database-backed state ensures consistency across restarts and multi-process deployments
• Connection-specific tuning: REST APIs, SSE streams, token refresh, and TLS tunnels each configure the shared breaker to match their reliability characteristics

Frequently Asked Questions

How does OmniRoute's circuit breaker differentiate between rate limits and other failures?

The classifyError callback maps errors to FailureKind values. For HTTP providers, status code 429 returns "rate_limit", 403 returns "quota_exhausted", and everything else defaults to "transient". Each kind can have its own threshold, cooldown, and immediateOpen flag in kindThresholds.

Can the circuit breaker state survive application restarts?

Yes. The breaker calls _persistToDb after state transitions and _restoreFromDb on initialization (lines 17-25 of src/shared/utils/circuitBreaker.ts). This ensures multi-process deployments and restarted instances share the same provider health view.

What happens when a circuit breaker reaches the OPEN state?

All execute() calls immediately throw CircuitBreakerOpenError without attempting the underlying operation. After the _effectiveResetTimeout expires, the breaker transitions to HALF_OPEN and allows exactly one probe request. Success closes the circuit; failure reopens it with extended back-off.

Why do TLS handshake failures trigger immediate OPEN instead of DEGRADED?

TLS handshake failures in open-sse/services/tlsClient/circuitBreaker.ts set immediateOpen: true because they indicate fundamental trust or connectivity problems—expired certificates, cipher mismatches, or potential interception—that won't resolve through gradual degradation. Instant isolation prevents sending data through compromised channels.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}