How OmniRoute Quota-Share Routing Distributes Shared Account Quotas: A Technical Deep Dive

OmniRoute quota-share routing distributes shared account quotas across multiple provider connections using a deterministic three-phase algorithm—bucket gating, deficit round-robin (DRR) ordering, and power-of-two-choices (P2C) selection—to ensure fair, weight-proportional traffic distribution while maintaining fail-open guarantees.

OmniRoute implements a specialized quota-share routing strategy for combos auto-minted with the qtSd/ prefix, providing in-process load distribution without database or network calls. This algorithm, maintained in the diegosouzapw/OmniRoute repository, executes on every request targeting a quota-share combo to balance load across shared account connections while respecting quota windows and real-time congestion.

The Three-Phase Distribution Algorithm

The quota-share routing algorithm processes each request through three ordered phases to determine the optimal target connection. Each phase filters or ranks candidates, with later phases refining the selection established by earlier ones.

Phase 1: Per-Model Bucket Gating

The algorithm first evaluates usage buckets for each connection to enforce quota limits. Every connection maintains three bucket types: a 5-hour window, a 7-day window, and a per-model 7-day window.

In open-sse/services/combo/quotaShareStrategy.ts, the filterEligibleBySaturation function calls isBucketSaturated from src/lib/quota/accountBuckets.ts to check these limits. If a bucket is saturated, the connection is de-prioritized (moved to the tail of the candidate list) but never dropped. This fail-open design ensures the combo always returns a candidate even when all connections have exhausted their quotas.

Phase 2: Deficit Round-Robin (DRR) Ordering

After gating, the remaining eligible connections are ordered using Deficit Round-Robin (DRR). This scheduling algorithm maintains a deficit counter for each target in an internal _drrState map.

The applyDrr function in quotaShareStrategy.ts adds a quantum proportional to each target's configured weight during every round. The target with the highest deficit is selected, and its deficit is reduced by 1. Over many requests, this yields deterministic, weight-proportional distribution that prevents starvation of lower-weight connections while honoring higher-weight allocations.

Phase 3: Power-of-Two-Choices (P2C) Load Balancing

The final selection occurs between the top two DRR candidates using Power-of-Two-Choices (P2C). The pickByInflightP2C function in open-sse/services/combo/quotaShareInflight.ts compares these candidates by their current in-flight request count.

The lighter-loaded connection wins the request; ties preserve the DRR winner. Once selected, incrementInflight immediately reserves capacity on the target, and the function returns a decrement callback for cleanup when the request completes. This prevents thundering-herd problems and balances real-time load across otherwise equivalent candidates.

Optional: Per-Connection Concurrency Caps

Before DRR processing, the algorithm optionally applies per-connection concurrency limits derived from provider_connections.max_concurrent. The partitionByConcurrencyCap function de-prioritizes connections that have reached their maxConcurrent limit behind those with available headroom.

Like bucket gating, this step respects the fail-open principle: at-cap connections are ranked lower but remain available if no alternatives exist.

Implementation Architecture

The routing strategy relies on four key files that isolate concerns between quota tracking, load balancing, and concurrency management:

• open-sse/services/combo/quotaShareStrategy.ts — Core orchestration implementing the three-phase algorithm (filterEligibleBySaturation, applyDrr, and main selection logic)
• src/lib/quota/accountBuckets.ts — Quota window calculations and isBucketSaturated checks
• open-sse/services/combo/quotaShareInflight.ts — Live request tracking with incrementInflight, decrementInflight, and lease-based expiration for P2C selection
• open-sse/services/combo/quotaShareConcurrency.ts — Semaphore-based concurrency enforcement used by partitionByConcurrencyCap

The final dispatch order follows this priority chain: the P2C winner, remaining connections with concurrency headroom, at-cap connections, and finally saturated (bucket-exhausted) connections.

Programming with Quota-Share Routing

You can invoke the selection logic programmatically using selectQuotaShareTarget for custom executors or testing scenarios:

import { selectQuotaShareTarget } from "@/open-sse/services/combo/quotaShareStrategy";
import type { ResolvedComboTarget } from "@/open-sse/services/combo/types";

// Example target list (normally built by the combo engine)
const targets: ResolvedComboTarget[] = [
  { connectionId: "c1", executionKey: "c1-0", weight: 2, provider: "anthropic", model: "claude-opus-4" },
  { connectionId: "c2", executionKey: "c2-0", weight: 1, provider: "openai", model: "gpt-4" },
];

// Optional concurrency caps from provider_connections.max_concurrent
const caps = new Map<string, number | null>([
  ["c1", 5],
  ["c2", null],  // null or <=0 indicates no limit
]);

const result = selectQuotaShareTarget(
  targets,
  "qtSd/my-shared-combo",      // combo name used as DRR state key
  "anthropic/claude-opus-4",   // requested model string
  Date.now(),
  { maxConcurrentByConnection: caps }
);

console.log("Chosen target:", result.target);
console.log("Full ordered list:", result.orderedTargets);

// Release the in-flight slot when the request completes
try {
  // Execute request against result.target...
} finally {
  result.decrementInflight();  // Idempotent cleanup
}

For deterministic testing, use the exported test utilities to control DRR state and timestamps:

import { selectQuotaShareTarget, _clearDrrStateForTest } from "@/open-sse/services/combo/quotaShareStrategy";

_clearDrrStateForTest();           // Reset state between tests
const now = 1722836400000;         // Fixed epoch milliseconds

const { target } = selectQuotaShareTarget(targets, "combo-test", "openai/gpt-4", now);
expect(target?.connectionId).toBe("c1"); // Deterministic outcome based on DRR weights

Fail-Open Guarantees and Edge Cases

Because every gating step is fail-open, a quota-share combo never hard-blocks a request even when all connections are saturated or at their concurrency limits. The algorithm guarantees that:

• Saturated connections are de-prioritized but remain selectable
• At-cap connections fall back to the tail of the queue
• A request always receives a candidate target regardless of system load

This design ensures high availability for quota-share scenarios where strict enforcement might otherwise cause total service denial during traffic spikes or quota exhaustion events.

Summary

• Three-phase algorithm: OmniRoute quota-share routing combines bucket gating, DRR ordering, and P2C selection to distribute shared account quotas fairly.
• Fail-open design: Connections are de-prioritized when buckets saturate or concurrency caps hit, but never removed from the candidate pool.
• Weight-proportional fairness: DRR ensures traffic distribution matches configured weights over time, while P2C prevents hotspots on currently busy connections.
• Zero external dependencies: The entire algorithm runs in-process without database queries or network calls, minimizing latency.
• Explicit resource management: In-flight tracking provides immediate capacity reservation and guaranteed cleanup via decrement callbacks.

Frequently Asked Questions

What distinguishes OmniRoute quota-share routing from simple round-robin?

Simple round-robin cycles through targets evenly regardless of configured weights or current load. OmniRoute quota-share routing uses Deficit Round-Robin to enforce weight-proportional distribution (higher weights receive proportionally more requests) and Power-of-Two-Choices to select the least-loaded candidate among the top weighted options, preventing hotspots on busy connections while maintaining fair allocation over time.

How does the algorithm behave when all connections have exhausted their quotas?

When all connections reach their bucket limits, the fail-open design in filterEligibleBySaturation moves all candidates to the de-prioritized tier but continues processing. The request proceeds through DRR and P2C phases using the saturated connections, ensuring the system remains operational. This prevents total service denial when shared accounts hit their limits, though operators may see increased latency or error rates from quota-exhausted providers.

Can quota-share routing handle dynamic concurrency limits during runtime?

Yes. The maxConcurrentByConnection parameter accepts a Map that can be updated between requests. The partitionByConcurrencyCap function evaluates these limits at selection time, dynamically de-prioritizing connections that have reached their current cap. Because the algorithm recalculates on every request, reducing a connection's maxConcurrent immediately affects subsequent routing decisions without requiring a system restart.

Why combine DRR with Power-of-Two-Choices instead of using a single load-balancing method?

DRR provides deterministic, weight-proportional fairness over time but lacks awareness of instantaneous load. P2C offers excellent real-time load balancing by comparing the two best candidates but ignores configured weights. Combining them—using DRR to select the top two weighted candidates, then P2C to pick the less busy one—achieves both goals: traffic distribution matches provider weights while avoiding thundering-herd problems on currently congested connections.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}