How OmniRoute Quota-Share Routing Distributes Requests Across Pooled API Keys

OmniRoute's quota-share routing automatically distributes requests across pooled API keys by creating a multi-step combo that serializes calls through each connection until it finds one with available quota, skipping exhausted keys without user intervention.

OmniRoute treats multiple API-key connections belonging to the same provider as a quota-share pool, enabling intelligent load distribution when traffic spikes exhaust individual keys. This internal routing strategy, implemented in the open-sse package, ensures high availability for pooled credentials by monitoring per-connection quota headers and enforcing concurrency limits. Understanding this mechanism helps you configure resilient multi-key setups that gracefully handle rate limits and quota exhaustion.

How Quota-Share Routing Works

The quota-share strategy operates through five distinct phases during request lifecycle management. Unlike round-robin or random selection, quota-share serializes execution across pool members, validating quota status before each attempt.

Pool Discovery and Metadata Storage

All connections sharing the same provider value and a common poolId are grouped into a quota-share pool. The pool metadata resides in src/lib/db/pool.ts, which defines the database schema for the pools table. When you assign multiple OpenAI keys (or any provider) to the same poolId, OmniRoute recognizes them as a collective resource subject to shared routing logic.

Combo Generation via syncQuotaCombos

The routing logic materializes in open-sse/services/combo.ts through the syncQuotaCombos() function. This routine:

1. Identifies every model available on all connections within the pool
2. Creates a combo with one step per connection pointing to that model
3. Sets combo.strategy = QUOTA_SHARE_STRATEGY (the constant "quota-share")

This internal-only strategy never appears in UI routing selectors; the platform mints these combos automatically when it detects pooled configurations.

Request Serialization and Quota Monitoring

When executing a request, the QuotaShareExecutor defined in open-sse/executors/quotaShare.ts processes the combo steps sequentially:

• Single-threaded execution: Only one connection attempt runs at a time
• Quota validation: Before each step, the executor checks the quota monitor (open-sse/services/quotaMonitor.ts) for quota_exhausted status
• HTTP status analysis: If a connection returns HTTP 403 or 429 with quota-related headers, the executor marks that key as exhausted for the current request
• Step skipping: The executor immediately proceeds to the next pooled connection without retrying the exhausted key

This serialization prevents the "thundering herd" problem where multiple simultaneous requests hit rate limits simultaneously.

Concurrency Guard Protection

Each pooled connection may specify a maxConcurrent limit. The quotaShareConcurrency.ts module in src/lib/resilience/ enforces these caps by queuing excess requests until active slots clear. This prevents 429 spikes caused by connection saturation before quota exhaustion even occurs.

Fallback Behavior

When all connections in a quota-share pool report exhausted quotas, the combo fails with the original quota error from the final attempted connection. Unlike multi-provider failover strategies, quota-share does not cascade to different providers—it strictly operates within the defined pool boundary. This behavior is verified by the unit tests in tests/unit/quota-share-strategy.test.ts.

Configuring Pooled API Keys for Quota-Share Routing

Implementing quota-share routing requires defining pools and connections in OmniRoute's database layer. The system automatically activates the strategy once it detects valid pool configurations.

Creating a Provider Pool

Define the pool metadata in src/lib/db/pool.ts before associating connections:

// Example pool creation via database insertion
await db.pools.insert({
  provider: "openai",
  poolId: "enterprise-production-pool",
  name: "Enterprise OpenAI Key Cluster",
  // Additional metadata: description, createdAt, etc.
});

Adding Connections to the Pool

Assign API keys to the pool using the poolId foreign key. Set optional concurrency limits to protect upstream providers:

await db.connections.insert({
  provider: "openai",
  apiKey: "sk-prod-key-1",
  poolId: "enterprise-production-pool",  // Links to the pool
  maxConcurrent: 10,                    // Enforces concurrency guard
});

await db.connections.insert({
  provider: "openai", 
  apiKey: "sk-prod-key-2",
  poolId: "enterprise-production-pool",
  maxConcurrent: 10,
});

Making Requests Through the Pool

Clients target models without specifying individual connections. OmniRoute resolves the appropriate quota-share combo automatically:

import { fetchChatCompletion } from "@omniroute/open-sse";

const response = await fetchChatCompletion({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Explain the quota-share routing algorithm" }],
  // Connection selection is handled internally by the combo resolver
});

If the first key in the pool has exhausted its monthly allocation, the executor transparently routes to the second key without exposing the failure to the client.

Debugging Combo Configuration

Inspect generated combos to verify quota-share assignment:

import { getComboByName } from "open-sse/services/combo";

const combo = await getComboByName("qtSd/openai/gpt-4o");
console.log(combo.strategy); // Expected: "quota-share"
console.log(combo.steps.map(s => s.connectionId)); 
// Output: Array of connection IDs participating in the pool

Error Handling and Edge Cases

Quota-share routing implements specific failure modes distinct from other OmniRoute strategies:

• Partial exhaustion: If 2 of 3 keys in a pool are exhausted, requests continue using the remaining key indefinitely until it also exhausts or the pool configuration changes
• Quota header parsing: The executor relies on the quota monitor's parsing of provider-specific headers (typically x-ratelimit-remaining or custom quota fields) to determine exhaustion state
• No cross-pool failover: As implemented in open-sse/executors/quotaShare.ts, quota-share combos do not fallback to other providers or non-pooled connections—they fail closed when the pool depletes

Summary

• Quota-share routing treats multiple API keys as a unified pool, distributing requests sequentially rather than randomly
• The syncQuotaCombos() function in open-sse/services/combo.ts generates these combos automatically when connections share a poolId
• The QuotaShareExecutor processes steps serially, checking open-sse/services/quotaMonitor.ts before each attempt to skip exhausted keys
• Concurrency limits are enforced via src/lib/resilience/quotaShareConcurrency.ts, preventing connection saturation
• When all pooled keys report quota exhaustion, the request fails with the last encountered quota error, verified by tests/unit/quota-share-strategy.test.ts

Frequently Asked Questions

What happens if one API key in the pool has a much lower rate limit than the others?

The QuotaShareExecutor checks quota status before every attempt, so keys with lower limits naturally receive fewer requests as they exhaust faster. However, because the strategy processes connections in fixed order rather than weighted distribution, you should ensure keys have roughly equivalent quotas for balanced utilization, or split mismatched keys into separate pools with different poolId values.

Can quota-share routing work across different providers (e.g., mixing OpenAI and Anthropic keys)?

No. Quota-share pools require all connections to share the same provider value, as implemented in src/lib/db/pool.ts and validated during combo generation in open-sse/services/combo.ts. Cross-provider failover requires different routing strategies configured explicitly in the UI or API, not the automatic quota-share mechanism.

How does OmniRoute detect that an API key's quota is exhausted?

The open-sse/services/quotaMonitor.ts service intercepts HTTP responses, looking for status codes 403 or 429 combined with provider-specific quota headers. When detected, it signals quota_exhausted to the QuotaShareExecutor, which immediately skips that connection for the current request and moves to the next step in the combo sequence.

Is there a performance penalty for using quota-share routing versus direct connection routing?

Yes, but it is minimal and intentional. The serialization in open-sse/executors/quotaShare.ts adds latency because requests execute sequentially rather than in parallel, and each step incurs a quota check against open-sse/services/quotaMonitor.ts. However, this prevents cascading failures from exhausted keys, typically resulting in higher overall success rates for pooled configurations under heavy load.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}