OmniRoute Circuit Breaker Lazy Recovery: Self-Healing Without Background Timers

OmniRoute implements lazy recovery by refreshing circuit breaker state only when status methods are called, eliminating background timers while ensuring providers transition from OPEN to HALF_OPEN precisely when cooldown periods expire.

OmniRoute protects its LLM routing pipeline from cascading failures using a sophisticated circuit breaker implementation. Unlike traditional approaches that rely on background scheduling threads, the CircuitBreaker class in src/shared/utils/circuitBreaker.ts employs a lazy recovery mechanism that checks state freshness exclusively on demand. This design reduces resource overhead while maintaining strict consistency between the breaker's internal state and observed behavior.

The Five-State Circuit Breaker Lifecycle

The implementation manages provider health through a strict state machine that includes an innovative intermediate status. According to the source code at lines 12–18, the breaker cycles through CLOSED → DEGRADED → OPEN → HALF_OPEN → CLOSED.

Degradation as an Early Warning

When the failure count exceeds a configurable degradation threshold (approximately 60% of the normal failure threshold), the breaker enters the DEGRADED state. This provides an early warning mechanism before the circuit fully opens, allowing the routing engine to deprioritize struggling providers without completely removing them from the pool.

Kind-Specific Failure Classification

Errors are classified by kind—including rate-limit, quota-exhausted, and transient failures. As implemented in lines 77–85, per-kind thresholds can trigger an immediate transition to OPEN, or apply custom cooldown periods based on the error semantics. This granularity prevents temporary rate limits from being treated identically to persistent authentication failures.

How Lazy Recovery Works

The lazy recovery mechanism eliminates the need for setInterval or background worker threads that periodically check timeout conditions. Instead, state transitions rely entirely on read-time evaluation.

On-Demand State Refreshing

The private method _refreshOpenState() (lines 48–52) performs all timeout checking. This method is invoked at the start of every public status query: canExecute(), getStatus(), and getRetryAfterMs(). When any component consults the breaker, the elapsed time since the last failure is compared against _effectiveCooldown(). If the timeout has not elapsed, the breaker remains OPEN. If the timeout has passed, the state transitions to HALF_OPEN immediately.

Transition to HALF_OPEN

When _shouldAttemptReset() confirms that elapsed time exceeds the effective cooldown, _refreshOpenState() calls _transition(STATE.HALF_OPEN, "timeout-elapsed"). The breaker then permits a limited number of probe requests controlled by the halfOpenRequests configuration parameter. A successful probe triggers _onSuccess(), which closes the circuit; a failure invokes _onFailure() and reopens the circuit, potentially with an increased back-off calculated by _effectiveResetTimeout().

SQLite Persistence for Process Survival

Breaker state—including last failure time, cumulative count, and error kind—is stored in the domain-state SQLite table via src/lib/db/domainState.ts. This persistence layer ensures that circuit status survives application restarts, preventing a restarted process from prematurely retrying a provider that triggered a cooldown just before termination.

Pipeline Integration and Usage Patterns

Components interact with the breaker through the getCircuitBreaker() factory, checking availability before executing upstream calls.

// Typical usage inside the request pipeline
const breaker = getCircuitBreaker(providerName);
if (!breaker.canExecute()) {
  // Provider is OPEN – skip it or fall back to another target
  throw new CircuitBreakerOpenError(
    `Provider ${providerName} circuit open`,
    providerName,
    breaker.getRetryAfterMs(),
  );
}

// Execute the LLM request
const response = await llmProvider.call(prompt);

// Record outcome
breaker.recordSuccess(); // internally calls _onSuccess()

The first line of canExecute() invokes this._refreshOpenState(), guaranteeing that any caller sees the most current state without additional scheduling overhead.

Observability via Health Endpoints

The lazy recovery mechanism enables real-time visibility through standard monitoring interfaces. The health route at src/app/api/v1/monitoring/health/route.ts exposes breaker status via the getStatus() method, which itself triggers state refresh.

// Lazy recovery in action – called via health endpoint
app.get("/api/v1/monitoring/health", async (req, res) => {
  const providers = Object.keys(providerRegistry);
  const breakerStatuses = providers.map((p) => getCircuitBreaker(p).getStatus());
  res.json({ providers: breakerStatuses });
});

Because status checks trigger _refreshOpenState(), the health endpoint always reports accurate recovery timing without polling or push-based updates.

Summary

• Lazy recovery eliminates background timers by checking cooldown expiration only when canExecute(), getStatus(), or getRetryAfterMs() are called.
• The five-state lifecycle includes a DEGRADED warning state triggered at approximately 60% of the failure threshold, preventing premature circuit opening.
• Kind-specific handling allows distinct thresholds and cooldowns for different error types such as rate limits or quota exhaustion.
• SQLite persistence in src/lib/db/domainState.ts maintains breaker state across process restarts.
• Health endpoint integration at src/app/api/v1/monitoring/health/route.ts provides real-time visibility into recovery status without additional monitoring infrastructure.

Frequently Asked Questions

What triggers the circuit breaker to move from OPEN to HALF_OPEN in OmniRoute?

The transition occurs when _refreshOpenState() detects that the effective cooldown period has elapsed during a status check. This method is invoked automatically when canExecute(), getStatus(), or getRetryAfterMs() are called, meaning the breaker recovers only when something actually queries its status rather than on a fixed timer.

How does OmniRoute's circuit breaker survive application restarts?

State is persisted to a SQLite database via src/lib/db/domainState.ts, which stores the last failure timestamp, cumulative failure count, and error kind. When the application restarts, the breaker rehydrates its state from this table, ensuring that cooldown periods continue to respect the original failure time.

Why doesn't OmniRoute use background timers for circuit breaker recovery?

Lazy recovery reduces CPU and memory overhead by avoiding setInterval or dedicated worker threads. Since _refreshOpenState() checks timeouts only on read operations, the system scales efficiently with thousands of provider instances without maintaining active timers for each breaker.

What is the DEGRADED state in OmniRoute's circuit breaker?

DEGRADED is an intermediate state entered when failure counts exceed approximately 60% of the configured failure threshold but haven't yet reached the full opening threshold. It signals to the routing engine that the provider is struggling, enabling preemptive load reduction before the circuit fully opens and blocks all traffic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}