# How OmniRoute Manages Model Lifecycle Policies: A Three-Layer Resilience System

> Discover how OmniRoute manages model lifecycle policies using its three-layer resilience system. Learn about provider circuit breaker, connection cooldown, and model lockout for dynamic governance.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-09-01

---

**OmniRoute v3.8.51 implements a three-layer resilience system—provider circuit breaker, connection cooldown, and model lockout—to dynamically govern model lifecycle policies from request validation through provider execution.**

OmniRoute's model lifecycle policies prevent cascading failures by isolating problems at different granularities. Rather than treating all errors equally, the system distinguishes between provider-wide outages, individual connection issues, and model-specific failures. This architecture ensures that a single bad API key or exhausted quota won't cripple the entire routing pipeline.

## Three-Layer Policy Architecture

OmniRoute organizes its lifecycle policies into three distinct layers, each targeting a different scope of failure.

| Layer | Scope | Purpose |
|-------|-------|---------|
| **Provider Circuit Breaker** | Entire provider (e.g., `openai`, `anthropic`) | Halts traffic to consistently failing upstreams |
| **Connection Cooldown** | Individual connection/account/key | Temporarily skips bad credentials while preserving other connections |
| **Model Lockout** | Provider + connection + specific model | Isolates per-model failures like quota limits or missing deployments |

The routing layer in [`open-sse/services/combo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo.ts) consults all three layers before selecting a target, automatically filtering out any provider, connection, or model that violates current policy state.

## Provider Circuit Breaker: System-Wide Protection

The **provider circuit breaker** stops sending traffic to an entire provider when upstream failures exceed configured thresholds. This protects OmniRoute from wasting resources on endpoints that are fundamentally unreachable.

### Implementation Details

The circuit breaker lives in [`src/shared/utils/circuitBreaker.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/utils/circuitBreaker.ts) and implements a four-state machine:

- `CLOSED` — Normal operation, requests allowed
- `DEGRADED` — Partial failure, limited traffic permitted
- `OPEN` — Circuit tripped, all requests blocked
- `HALF_OPEN` — Testing recovery with a single probe request

Configuration resides in `PROVIDER_PROFILES` within [`open-sse/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/constants.ts). The [`src/sse/handlers/chatHelpers.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/sse/handlers/chatHelpers.ts) module applies these profiles during request validation.

```typescript
// Checking if a provider is allowed to execute
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';
import { Provider } from '@/shared/constants/providers';

async function canExecute(provider: Provider): Promise<boolean> {
  const breaker = getCircuitBreaker(provider);
  // State refreshes lazily from timestamps on each read
  return breaker.canExecute(); // true if CLOSED/DEGRADED, false if OPEN
}

```

The breaker uses **lazy state evaluation**—timestamps like `rateLimitedUntil` are checked on every `canExecute()` call rather than relying on background timers. This eliminates coordination overhead while enabling automatic recovery.

## Connection Cooldown: Granular Credential Isolation

When a specific API key or account hits rate limits, the **connection cooldown** layer marks that individual connection unavailable without affecting other connections to the same provider.

### Key Implementation Files

- `src/sse/services/auth.ts::markAccountUnavailable()` — triggers cooldown after failure detection
- `open-sse/services/accountFallback.ts::checkFallbackError()` — validates cooldown status before routing
- [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) — stores cooldown durations and retry policies

```typescript
// Marking an account unavailable after a 429 response
import { markAccountUnavailable } from '@/sse/services/auth';

async function handleRateLimited(err: any, accountId: string) {
  if (err.status === 429) {
    await markAccountUnavailable(accountId, {
      retryAfterMs: err.retryAfter ?? 3000, // 3-second fallback
    });
  }
}

```

Cooldown state persists as timestamps, allowing automatic expiration without scheduled jobs. A connection becomes eligible for retry once its `retryAfterMs` period elapses.

## Model Lockout: Fine-Grained Failure Isolation

The most granular layer, **model lockout**, handles failures specific to individual models within a connection—such as per-model quota exhaustion or deployments that no longer exist.

### Lockout Logic

Implemented in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts), model lockout prevents the routing layer from selecting compromised models while preserving healthy ones on the same connection. The combo routing logic consults this state before finalizing target selection.

```typescript
// Disabling a specific model after quota exhaustion
import { disableModelLockout } from '@/open-sse/services/accountFallback';

async function handleModelQuota(err: any, modelId: string, connectionId: string) {
  if (err.status === 429 && err.model === modelId) {
    await disableModelLockout(connectionId, modelId);
  }
}

```

Model lockout records per-model error flags that persist independently of connection cooldown state. This means a `429` on `gpt-4-turbo` won't disable `gpt-4o` on the same credential.

## Policy Engine Integration

Higher-level services interact with lifecycle policies through [`src/domain/policyEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/policyEngine.ts), which abstracts the three layers into a unified query interface. This abstraction allows routing components to check policy compliance without directly managing state machines or timestamp arithmetic.

The policy engine's lazy evaluation design ensures **zero background overhead**—no timers, no polling loops, no goroutines. State transitions happen inline with request processing, and expiration is computed on demand.

## Thresholds and Configuration

Default thresholds live in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) and can be overridden via environment variables:

| Failure Type | Typical Threshold | Action |
|-------------|-------------------|--------|
| Provider HTTP 5xx | 5 errors in 60 seconds | Circuit breaker → OPEN |
| Connection 429 | 3 consecutive rate limits | Cooldown for `retryAfter` duration |
| Model-specific 429 | 1 error with model attribution | Model lockout for 5 minutes |

These defaults balance rapid failure detection against unnecessary isolation of transient issues.

## Code Examples: Complete Policy Flow

```typescript
// Full request pipeline with policy checks
import { getCircuitBreaker } from '@/shared/utils/circuitBreaker';
import { checkFallbackError } from '@/open-sse/services/accountFallback';
import { selectComboTarget } from '@/open-sse/services/combo';

async function routeRequest(request: ChatRequest) {
  // Layer 1: Provider-level check
  const providerBreaker = getCircuitBreaker(request.provider);
  if (!await providerBreaker.canExecute()) {
    throw new ProviderUnavailableError(request.provider);
  }

  // Layer 2 & 3: Connection and model checks happen inside combo routing
  const target = await selectComboTarget(request, {
    filterCooldown: true,
    filterLockedModels: true,
  });

  if (!target) {
    throw new NoAvailableModelsError();
  }

  return target;
}

```

The `selectComboTarget` function encapsulates the interaction between connection cooldown and model lockout layers, returning only targets that satisfy all active policies.

## Summary

OmniRoute's model lifecycle policy system provides:

- **Three isolation layers** protecting against provider, connection, and model-level failures
- **Lazy state evaluation** eliminating background timers and enabling automatic recovery
- **Configurable thresholds** via [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts) and environment variables
- **Unified policy engine** abstraction in [`src/domain/policyEngine.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/domain/policyEngine.ts) for clean integration

These mechanisms ensure that upstream outages, rate limits, and model-specific issues are contained without manual intervention or global service degradation.

## Frequently Asked Questions

### What triggers a provider circuit breaker to open?

The provider circuit breaker transitions to `OPEN` when HTTP 5xx errors exceed the threshold defined in `PROVIDER_PROFILES`—typically 5 errors within 60 seconds. Once open, the breaker blocks all traffic to that provider until the `HALF_OPEN` probe succeeds or the timeout expires.

### How does OmniRoute recover from a connection cooldown automatically?

Connection cooldowns store `rateLimitedUntil` timestamps that are checked lazily on each routing decision. No background process monitors expiration; instead, `checkFallbackError()` in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts) validates whether the cooldown period has elapsed before filtering out a connection.

### Can model lockout affect other connections using the same provider?

No. Model lockout is scoped to a specific `connectionId + modelId` pair as implemented in [`open-sse/services/accountFallback.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/accountFallback.ts). A `gpt-4` lockout on one API key does not impact `gpt-4` availability on other credentials or providers.

### Where are default resilience thresholds configured?

Default thresholds for all three policy layers reside in [`src/lib/resilience/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/resilience/settings.ts), with provider-specific profiles in [`open-sse/config/constants.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/config/constants.ts). These values can be overridden through environment variables without code changes.