OmniRoute API Key Policies: Authentication, Rate Limiting, and Circuit Breakers

OmniRoute enforces a hierarchical three-layer API key policy that mandates authentication via the REQUIRE_API_KEY environment variable, applies exponential backoff cooldowns to individual failing keys, and escalates to provider-wide circuit breakers when failure thresholds exceed configurable limits.

OmniRoute is an open-source routing engine designed to manage traffic across multiple LLM providers. The API key policies in OmniRoute govern how client credentials are validated, rate-limited, and protected through resilient failover mechanisms. According to the diegosouzapw/OmniRoute source code, this layered architecture ensures that transient failures on individual keys cannot cascade into total provider outages.

The Three-Layer Policy Architecture

The API key policy operates through three tightly integrated layers that range from global request gating to granular connection management.

Global Enforcement Layer

The foundation of the policy rests on the REQUIRE_API_KEY environment variable, which defaults to true in production environments. When enabled, every inbound request must present a valid Authorization header before any routing logic executes.

In src/app/api/v1/auth/route.ts, the middleware parses headers formatted as Bearer <key> or Token <key> and validates the token against the API_KEY_SECRET hash stored in the database layer (src/lib/db/apiKeys.ts). If the variable is explicitly set to false, the enforcement check is bypassed entirely.

Connection-Level Cooldown

When a specific key encounters transient failures—such as HTTP 429 (Too Many Requests) or 5xx server errors—OmniRoute places that connection into a cooldown state defined by the rateLimitedUntil timestamp. This mechanism prevents the routing engine from sending additional requests to failing endpoints until the timeout expires.

The cooldown logic in src/sse/services/auth.ts implements exponential backoff using the formula baseCooldownMs * 2**failureIndex, where the base interval and maximum retry parameters are configured in src/lib/resilience/settings.ts.

Provider-Wide Circuit Breaker

If multiple keys associated with the same provider repeatedly fail, the system escalates to a circuit breaker pattern defined in src/shared/utils/circuitBreaker.ts. When the failure count exceeds the providerFailureThreshold—defaults of 10 for OAuth providers and 15 for API-key providers as specified in src/shared/constants/providers.ts—the entire provider is marked as OPEN.

While open, all keys for that provider are excluded from routing for a configurable PROVIDER_COOLDOWN_MS period (5 minutes for OAuth, 10 minutes for API-key providers). The breaker automatically transitions to HALF-OPEN to test recovery before fully closing.

API Key Validation and Scope Enforcement

Beyond basic authentication, the policy enforces granular access controls through signature validation and scoping rules.

Signature Verification

The raw API key is hashed using the API_KEY_SECRET (generated during setup via .env.example) and looked up in the keys table managed by src/lib/db/apiKeys.ts. Requests presenting invalid signatures receive an immediate 403 Forbidden response.

Scope and Quota Management

Each key can be restricted to specific providers, models, or provider-model combinations stored in the api_key_scopes table. During request processing, the policyEngine in src/domain/policyEngine.ts evaluates connectionId, apiKeyId, and comboId to apply reasoning rules and enforce quota limits before forwarding traffic to upstream providers.

Rate Limiting and Resilience Configuration

The resilience settings are centralized to ensure consistent behavior across the routing layer.

Parameter	Default Value	Location
providerFailureThreshold (OAuth)	10	src/shared/constants/providers.ts
providerFailureThreshold (API Key)	15	src/shared/constants/providers.ts
PROVIDER_COOLDOWN_MS (OAuth)	5 minutes	src/shared/constants/providers.ts
PROVIDER_COOLDOWN_MS (API Key)	10 minutes	src/shared/constants/providers.ts
Backoff multiplier	2^failureIndex	src/lib/resilience/settings.ts

This hierarchical design guarantees that a single misbehaving key triggers only a connection cooldown, while widespread provider issues trigger the circuit breaker, preventing cascading failures.

Implementation Examples

Configure the environment variables to enable strict authentication:

# .env configuration

REQUIRE_API_KEY=true
API_KEY_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Issue requests with proper authorization headers:

fetch('https://localhost:20128/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.MY_API_KEY}`
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello' }]
  })
});

The authentication middleware extracts and validates tokens before routing:

import { getApiKeyRecord } from '@/lib/db/apiKeys';
import { buildErrorBody } from '@/open-sse/utils/error';

export async function authMiddleware(req) {
  if (process.env.REQUIRE_API_KEY !== 'false') {
    const token = req.headers.get('authorization')?.split(' ')[1];
    
    if (!token) {
      return { status: 401, body: buildErrorBody('API key missing') };
    }
    
    const keyRec = await getApiKeyRecord(token);
    if (!keyRec) {
      return { status: 403, body: buildErrorBody('Invalid API key') };
    }
    
    // Attach the key ID for downstream routing decisions
    req.apiKeyId = keyRec.id;
  }
  return next();
}

Connection-level cooldown logic applies exponential backoff on rate limit errors:

if (errorCode === 429) {
  const backoffMs = BASE_COOLDOWN_MS * 2 ** connection.failureCount;
  connection.rateLimitedUntil = Date.now() + backoffMs;
}

Summary

• Global Enforcement: The REQUIRE_API_KEY variable in src/app/api/v1/auth/route.ts mandates authentication for all requests unless explicitly disabled.
• Connection Cooldown: Individual keys entering a failure state are temporarily bypassed via rateLimitedUntil timestamps with exponential backoff defined in src/sse/services/auth.ts.
• Circuit Breaker Protection: Provider-wide failures trigger automatic exclusion periods (5-10 minutes) managed by src/shared/utils/circuitBreaker.ts when providerFailureThreshold limits are exceeded.
• Scope Validation: The policy engine in src/domain/policyEngine.ts enforces provider and model restrictions using the api_key_scopes database table.
• Hierarchical Precedence: Provider circuit breakers supersede individual key cooldowns, ensuring system-wide protection takes priority over per-connection management.

Frequently Asked Questions

How do I disable API key authentication in OmniRoute?

Set the environment variable REQUIRE_API_KEY=false in your .env file or deployment configuration. When disabled, the authentication check in src/app/api/v1/auth/route.ts is bypassed, allowing unauthenticated requests to reach the routing engine. This setting defaults to true and should only be disabled in trusted internal networks or development environments.

What happens when an API key receives a 429 Too Many Requests response?

The key enters a connection cooldown state where the rateLimitedUntil field is populated with a future timestamp calculated using exponential backoff. During this period, the routing engine in src/sse/services/auth.ts skips this key for new requests. Once the timestamp expires, the key becomes eligible for selection again, with the failureCount reset upon successful requests.

How does the provider circuit breaker differ from individual key cooldowns?

The connection-level cooldown affects only the specific key that encountered errors, while the provider circuit breaker in src/shared/utils/circuitBreaker.ts disables the entire provider when aggregate failures across multiple keys exceed the providerFailureThreshold. Individual cooldowns use exponential backoff calculated in src/lib/resilience/settings.ts, whereas circuit breakers use fixed PROVIDER_COOLDOWN_MS durations (5 or 10 minutes depending on provider type).

Where are API key permissions and provider restrictions stored?

Key metadata and hashed secrets reside in the database table managed by src/lib/db/apiKeys.ts, while granular scope restrictions—defining which providers and models a key may access—are stored in the api_key_scopes table. The src/domain/policyEngine.ts module queries these tables during request routing to enforce comboId and connectionId level policies before forwarding to upstream LLM endpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}