How the Quota-Share Combo Works in OmniRoute: Multi-Account Routing with Per-Connection Limits

OmniRoute’s quota-share combo distributes a single logical request across multiple provider connections, enforcing individual quota limits and concurrency caps while automatically retrying on quota errors.

The quota-share combo is a specialized routing strategy in the open-source OmniRoute proxy that enables cost optimization and reliability when working with multiple API provider accounts. Unlike standard round-robin routing, this mechanism treats a pool of accounts as a unified endpoint while preserving each connection’s quota constraints, making it ideal for high-volume applications that need to stay within free-tier limits across several keys.

Core Architecture and Key Files

The implementation spans the backend service layer and the React dashboard UI. Understanding the file structure helps trace how a request flows from detection to execution:

• open-sse/services/combo.ts – Central routing entry point that detects combo types and delegates to specialized handlers
• open-sse/services/combo/quotaShareStrategy.ts – Implements the selection algorithm (DRR-style) for choosing which connection handles the request
• open-sse/services/combo/quotaShareConcurrency.ts – Enforces per-connection concurrency limits using semaphores
• open-sse/services/combo/comboCooldownRetry.ts – Handles cooldown periods and retry logic after quota errors
• src/app/(dashboard)/dashboard/costs/quota-share/QuotaSharePageClient.tsx – Dashboard UI for monitoring usage and configuring limits

Step-by-Step Request Flow

When OmniRoute receives a request designated for a quota-share combo, it executes a seven-stage pipeline that balances load while respecting provider constraints.

1. Combo Detection via Prefix Matching

The routing engine identifies quota-share combos by checking for the prefix qtSd/ in the combo ID. This occurs in the main combo service file before any target resolution begins.

// open-sse/services/combo.ts (around line 805)
if (comboId.startsWith('qtSd/')) {
  return this.handleQuotaShareCombo(comboId, request);
}

2. Target Expansion

The resolveComboTargets() function expands the combo ID into an ordered list of ResolvedComboTarget objects. Each target represents a distinct provider connection with its own quota counters and concurrency settings.

// open-sse/services/combo.ts (around line 2057)
const targets: ResolvedComboTarget[] = await this.resolveComboTargets(comboId);
// Each target contains connectionId, remainingQuota, maxConcurrent, etc.

3. Connection Selection Strategy

The quota-share strategy (quotaShareStrategy.ts) implements a DRR-like (Deficit Round Robin) algorithm that prefers connections with the most remaining quota. Unlike strict load balancing, this approach ensures optimal utilization of available quota capacity across the pool.

// open-sse/services/combo/quotaShareStrategy.ts
function selectConnection(targets: ResolvedComboTarget[]): ResolvedComboTarget {
  // Sort by remaining quota (descending) and select best candidate
  const available = targets.filter(t => t.remainingQuota > 0);
  return available.sort((a, b) => b.remainingQuota - a.remainingQuota)[0];
}

4. Per-Connection Concurrency Guard

For quota-share combos, OmniRoute enforces a Max Concurrent limit per connection using a semaphore implementation in quotaShareConcurrency.ts. When a connection reaches its limit, additional requests queue rather than receiving a 429 error, protecting upstream APIs from burst traffic that could exhaust quota limits prematurely.

// open-sse/services/combo/quotaShareConcurrency.ts
class QuotaShareConcurrency {
  private semaphores = new Map<string, Semaphore>();
  
  async acquire(connectionId: string, maxConcurrent: number): Promise<void> {
    if (!this.semaphores.has(connectionId)) {
      this.semaphores.set(connectionId, new Semaphore(maxConcurrent));
    }
    await this.semaphores.get(connectionId)!.acquire();
  }
}

5. Cooldown-Aware Error Handling

When an upstream provider returns a quota error (HTTP 402 or 403), the comboCooldownRetry.ts module determines whether to pause and retry with the next connection. This graceful degradation ensures requests complete successfully even when individual accounts hit their limits.

// open-sse/services/combo/comboCooldownRetry.ts
function shouldCooldownRetry(error: ProviderError): boolean {
  return error.statusCode === 402 || error.statusCode === 403;
}

async function executeWithCooldown(targets: ResolvedComboTarget[], request: Request) {
  for (const target of targets) {
    try {
      return await executeRequest(target, request);
    } catch (error) {
      if (shouldCooldownRetry(error) && target.hasNext) {
        await delay(calculateCooldown(error));
        continue;
      }
      throw error;
    }
  }
}

6. Execution Handoff

Once a connection is selected and concurrency permits, the combo layer hands off to handleSingleModel() (line 226 in combo.ts), which performs standard request translation and header construction. The combo layer only influences which connection executes the request and when retries occur.

7. Dashboard Monitoring

The React dashboard (QuotaSharePageClient.tsx) exposes real-time metrics for each connection, including remaining quota, active concurrent requests, and cooldown status. Administrators can toggle the per-connection concurrency guard or adjust Max Concurrent caps without restarting the service.

Practical Configuration Example

To configure a quota-share combo in OmniRoute, define multiple connections under a single combo ID with the qtSd/ prefix:

{
  "comboId": "qtSd/production-pool",
  "strategy": "quota-share",
  "connections": [
    {
      "id": "openai-account-1",
      "apiKey": "sk-...",
      "maxConcurrent": 10,
      "quotaLimit": 1000
    },
    {
      "id": "openai-account-2", 
      "apiKey": "sk-...",
      "maxConcurrent": 5,
      "quotaLimit": 1000
    }
  ]
}

The routing engine automatically balances traffic to maximize the combined 2000-request quota while ensuring neither account exceeds its individual 10 or 5 concurrent request limits.

Summary

• Quota-share combos use the qtSd/ prefix to enable multi-account routing with individual quota enforcement.
• DRR-style selection in quotaShareStrategy.ts prioritizes connections with the most remaining capacity.
• Per-connection semaphores in quotaShareConcurrency.ts prevent bursts from exhausting quotas early by queueing excess requests.
• Automatic failover via comboCooldownRetry.ts handles HTTP 402/403 errors by retrying with alternate connections after brief cooldowns.
• Dashboard visibility through QuotaSharePageClient.tsx provides real-time monitoring of distributed quota consumption.

Frequently Asked Questions

How does OmniRoute handle it when all connections in a quota-share combo hit their limits?

When every connection returns a quota error (HTTP 402/403), the comboCooldownRetry.ts module exhausts its retry loop and propagates the final error to the client. However, before giving up, it applies staggered cooldown delays between attempts, maximizing the chance that a connection with a rolling quota window becomes available during the retry sequence.

What is the difference between the quota-share combo and standard load balancing?

Standard load balancing distributes requests evenly regardless of remaining capacity, while the quota-share combo specifically tracks per-connection quota counters and applies a DRR-like algorithm to prefer under-utilized connections. Additionally, quota-share implementations enforce per-connection concurrency caps using semaphores—a feature absent from standard routing strategies in OmniRoute.

Can I mix different provider types within a single quota-share combo?

Yes, the resolveComboTargets() function treats each ResolvedComboTarget as an abstraction, allowing you to pool connections from OpenAI, Anthropic, or other providers under a single qtSd/ combo ID. The strategy layer selects based on quota availability and concurrency status without regard to the underlying provider, though you should ensure compatible request schemas in the configuration.

Where does the concurrency semaphore actually block requests?

The blocking occurs in quotaShareConcurrency.ts before the request reaches handleSingleModel(). When a connection's active request count equals its Max Concurrent setting, subsequent requests await in the semaphore queue until an in-flight request completes, effectively smoothing traffic spikes that would otherwise trigger rate limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}