How OmniRoute's Circuit Breaker Differentiates Provider-Level vs Account-Level Errors

OmniRoute uses a two-layer resilience system: provider-level circuit breakers block entire upstream providers when systemic failures occur, while account-level cooldowns isolate individual credentials when only one API key or token fails.

The diegosouzapw/OmniRoute routing engine implements granular fault tolerance that prevents a single bad credential from taking down an entire provider, while also protecting against thundering-herd scenarios during upstream outages. This dual-layer approach lives primarily in src/shared/utils/circuitBreaker.ts and open-sse/services/accountFallback.ts.

Provider-Level Circuit Breakers: Protecting Against Systemic Failures

OmniRoute aggregates failures across all connections to a provider using the CircuitBreaker class. When failure rates exceed configured thresholds, the entire provider is temporarily blocked.

How Provider Breakers Are Triggered

In open-sse/services/accountFallback.ts, errors are classified by HTTP status code. Only failures matching PROVIDER_FAILURE_ERROR_CODES (defined at lines 226-227) propagate to the provider-level breaker:

408 (Request Timeout), 429 (Rate Limit), and 500-504 (Server Errors)

These indicate infrastructure problems rather than credential-specific issues.

// From open-sse/services/accountFallback.ts
const PROVIDER_FAILURE_ERROR_CODES = [408, 429, 500, 501, 502, 503, 504];

The breaker singleton is retrieved via getCircuitBreaker(provider) and updated through CircuitBreaker.execute() in the request pipeline.

Deduplication Prevents Inflation

To avoid skewing provider breaker metrics from rapid retries on the same failing connection, OmniRoute maintains two deduplication windows:

• lastConnectionFailure: 5-second window per connection
• lastNetworkErrorByProvider: 10-second window per provider

Implemented at lines 28-41 of accountFallback.ts, these ensure that a single network blip doesn't immediately open the provider breaker, while still capturing genuinely distributed failures.

State Transitions in circuitBreaker.ts

The CircuitBreaker class manages three states:

• CLOSED: Normal operation, failures tracked
• OPEN: All requests short-circuited, canExecute() returns false
• HALF_OPEN: Testing recovery with limited traffic

State transitions occur in _onFailure(kind) (lines 94-106 of circuitBreaker.ts). When failureThreshold is exceeded, the breaker moves to OPEN for resetTimeout milliseconds.

Account-Level Cooldowns: Isolating Individual Credentials

When an error doesn't indicate systemic provider failure—such as 401 Unauthorized, 403 Forbidden, or model-specific quota exhaustion—OmniRoute handles it at the connection level.

Connection State Tracking

Per-account resilience is configured through ProviderProfile fields in accountFallback.ts (lines 20-30):

Field	Purpose
baseCooldownMs	Initial backoff duration for this account
maxBackoffSteps	Maximum exponential backoff iterations
rateLimitCooldown	Specific duration for 429 responses

The Account Availability Pipeline

1. Error classification: Functions like classify429FromError categorize failures
2. State recording: markAccountUnavailable writes rateLimitedUntil, backoffLevel, and lastError to the database
3. Execution check: canExecute in request handlers queries isAccountUnavailable (lines 41-44) to skip cooled connections
4. Recovery: recordProviderSuccess clears cooldowns when requests succeed

// Conceptual flow from accountFallback.ts
if (errorStatus === 429 && error.code === 'insufficient_quota') {
  // Account-level: only this API key is affected
  await markAccountUnavailable(connectionId, 'quota_exhausted', backoffMs);
} else if (PROVIDER_FAILURE_ERROR_CODES.includes(errorStatus)) {
  // Provider-level: contributes to aggregate circuit breaker
  getCircuitBreaker(provider).recordFailure();
}

Decision Flow: Where Errors Get Routed

Incoming Error
    │
    ▼
┌─────────────────┐
│ HTTP Status Code│
│   Classification │
└────────┬────────┘
         │
    ┌────┴────┐
    ▼         ▼
408,429,5xx  401,403,quota
(PROVIDER)   (ACCOUNT)
    │         │
    ▼         ▼
┌─────────┐ ┌─────────────────┐
│Deduplicate│ │markAccountUnavailable│
│  (5s/10s) │ │  update DB row       │
└────┬────┘ └─────────────────┘
     │
     ▼
┌─────────────────┐
│getCircuitBreaker│
│  (provider)     │
│  .recordFailure()│
└────────┬────────┘
         ▼
    Threshold Check
    (failureThreshold)
         │
    ┌────┴────┐
    ▼         ▼
   CLOSED    THRESHOLD EXCEEDED
              │
              ▼
            OPEN
   (all provider requests blocked)

Configuration and Operational Controls

Inspecting Breaker Status

import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";

const cb = getCircuitBreaker("openai");
console.log(cb.getStatus());
// {
//   name: "openai",
//   state: "CLOSED",
//   failureCount: 0,
//   retryAfterMs: 0,
//   ...
// }

Checking Individual Account Availability

import { getProviderCredentials } from "@omniroute/open-sse/services/auth";

const creds = await getProviderCredentials("openai", null, null, null, {
  allowRateLimitedConnections: false,
});
if (!creds) {
  console.log("All OpenAI accounts are currently rate‑limited or unavailable");
}

Emergency Reset

import { resetAllCircuitBreakers } from "@/shared/utils/circuitBreaker";

resetAllCircuitBreakers(); // Clears provider-level state post-deployment

Key Source Files

File	Responsibility	Link
src/shared/utils/circuitBreaker.ts	Generic breaker implementation, provider registry, persistence	circuitBreaker.ts
open-sse/services/accountFallback.ts	Error classification, deduplication, account cooldown logic	accountFallback.ts
src/lib/resilience/settings.ts	Default breaker profiles (failureThreshold, resetTimeout)	settings.ts
src/lib/monitoring/providerHealthMatrix.ts	Health API exposure of breaker states	providerHealthMatrix.ts

Summary

• Provider-level circuit breakers in circuitBreaker.ts protect against upstream infrastructure failures using aggregated failure counts across all connections, triggered by HTTP 408/429/5xx status codes with 5-10 second deduplication windows.

• Account-level cooldowns in accountFallback.ts isolate individual credentials through database-tracked rateLimitedUntil timestamps and exponential backoff, preserving healthy accounts when one API key fails.

• Dual-layer classification happens at error time: systemic codes hit the provider breaker, credential-specific issues trigger per-account throttling—ensuring neither noisy neighbors nor upstream outages degrade routing availability.

Frequently Asked Questions

How do I know if a provider is in OPEN state versus just having rate-limited accounts?

Check the provider breaker directly via getCircuitBreaker(provider).getStatus(). When state equals "OPEN", all accounts are blocked. If state is "CLOSED" but requests still fail, individual accounts likely have active cooldowns—query isAccountUnavailable for specific connection IDs.

Can I adjust the failure threshold for a specific provider?

Yes. Pass a custom profile when initializing the breaker: getCircuitBreaker("anthropic", { failureThreshold: 5, resetTimeout: 30000 }). The default profiles are defined in src/lib/resilience/settings.ts and applied at accountFallback.ts lines 84-89.

Why does OmniRoute deduplicate provider failures instead of counting every error?

The 5-second lastConnectionFailure and 10-second lastNetworkErrorByProvider windows prevent scenario where rapid retries on one bad connection artificially inflate the aggregate failure rate. This ensures the provider breaker genuinely reflects distributed outages, not localized retry storms.

What happens when a provider breaker is OPEN but some accounts still have quota?

All traffic to that provider is rejected regardless of individual account health. This is intentional: if OpenAI returns 503s across multiple accounts, attempting alternate credentials wastes latency and may exacerbate the upstream degradation. Accounts resume eligibility once the breaker transitions to HALF_OPEN after resetTimeout.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}