How OmniRoute Quota-Share Routing Distributes Shared Account Quota Across Connections

OmniRoute distributes shared account quota through a three-phase in-process algorithm: bucket-gating checks, Deficit Round-Robin weighted ordering, and Power-of-Two-Choices load balancing—always failing open so no request is hard-blocked.

The quota-share routing strategy in OmniRoute handles combos auto-minted with the qtSd/ prefix. Unlike other routing modes, this implementation is entirely in-process: no database queries or network calls occur during target selection. This design ensures sub-millisecond overhead while distributing load fairly across multiple provider connections sharing a single quota pool.

Three-Phase Quota Distribution Algorithm

The algorithm runs on every request targeting a quota-share combo. Each phase refines the candidate list while maintaining the fail-open guarantee—a request always has at least one candidate, even when all connections are saturated.

Phase 1: Per-Model Bucket Gating

Each connection tracks usage buckets for three windows: 5-hour, 7-day, and per-model 7-day. The filterEligibleBySaturation function in [quotaShareStrategy.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/quotaShareStrategy.ts) checks these buckets via isBucketSaturated from [accountBuckets.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/quota/accountBuckets.ts).

• Saturated connections are de-prioritized (moved to the tail of the candidate list)
• Never dropped—the combo remains viable for every request
• Evaluation happens against the requested model string (e.g., "anthropic/claude-opus-4")

This fail-open approach distinguishes quota-share routing from hard-rejection strategies. Even when quota windows are exhausted, traffic continues to flow through de-prioritized channels.

Phase 2: Deficit Round-Robin (DRR) Ordering

On the remaining eligible connections, applyDrr implements weighted fair queuing via per-target deficit tracking:

DRR Component	Behavior
Quantum	Added each round, proportional to connection weight
Deficit accumulator	Each target maintains running deficit in _drrState
Selection	Highest deficit wins; winner's deficit decremented by 1
Outcome	Deterministic, weight-proportional distribution over time

The DRR state persists in-memory using the combo name as key. For combo qtSd/my-shared-combo, deficits accumulate independently of other combos sharing the same connections.

Phase 3: Power-of-Two-Choices (P2C) Load Balancing

The final selection occurs in pickByInflightP2C from [quotaShareInflight.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/quotaShareInflight.ts):

1. Take top two DRR candidates
2. Compare their current in-flight request counts
3. Select the lighter-loaded candidate
4. Tie-breaker: DRR winner prevails

Chosen targets receive immediate in-flight increment; a cleanup callback decrements on completion. This prevents thundering-herd scenarios without centralized coordination.

Optional Phase: Per-Connection Concurrency Cap

Before DRR execution, partitionByConcurrencyCap evaluates provider_connections.max_concurrent:

• Connections at their cap → de-prioritized behind connections with headroom
• null or ≤0 values → treated as unlimited
• Maintains fail-open: at-cap connections remain viable, just ordered later

Final Dispatch Priority

The complete ordering after all phases:

1. P2C winner (lightest-loaded of top-2 DRR)
2. Connections with concurrency headroom
3. Connections at concurrency cap
4. Bucket-saturated connections (de-prioritized)

This hierarchy ensures quota-respecting weighted distribution while preserving service availability under all conditions.

Code Implementation Examples

Programmatic Target Selection

import { selectQuotaShareTarget } from "@/open-sse/services/combo/quotaShareStrategy";
import type { ResolvedComboTarget } from "@/open-sse/services/combo/types";

const targets: ResolvedComboTarget[] = [
  { connectionId: "c1", executionKey: "c1-0", weight: 2, provider: "anthropic", model: "claude-opus-4" },
  { connectionId: "c2", executionKey: "c2-0", weight: 1, provider: "openai", model: "gpt-4" },
];

const caps = new Map<string, number | null>([
  ["c1", 5],
  ["c2", null],
]);

const result = selectQuotaShareTarget(
  targets,
  "qtSd/my-shared-combo",
  "anthropic/claude-opus-4",
  Date.now(),
  { maxConcurrentByConnection: caps }
);

// Execute request against result.target
try {
  await executeRequest(result.target);
} finally {
  result.decrementInflight();
}

Deterministic Testing

import { selectQuotaShareTarget, _clearDrrStateForTest } from "@/open-sse/services/combo/quotaShareStrategy";

_clearDrrStateForTest();
const now = 1722836400000;

const { target } = selectQuotaShareTarget(targets, "combo-test", "openai/gpt-4", now);
// DRR outcome is deterministic with fixed clock and clean state

Key Source Files

File	Responsibility
open-sse/services/combo/quotaShareStrategy.ts	Core three-phase orchestration
src/lib/quota/accountBuckets.ts	Bucket saturation detection
open-sse/services/combo/quotaShareInflight.ts	In-flight tracking and P2C selection
open-sse/services/combo/quotaShareConcurrency.ts	Per-connection semaphore management
tests/unit/quota-share-strategy.test.ts	Algorithm verification and fail-open assertions

Summary

• Quota-share routing operates entirely in-process with zero external dependencies
• Three ordered phases (bucket gating → DRR → P2C) progressively refine target selection
• Fail-open design guarantees at least one candidate per request regardless of quota state
• Weighted distribution via Deficit Round-Robin respects configured connection weights
• Load awareness through Power-of-Two-Choices in-flight comparison prevents hot-spotting
• Concurrency caps provide optional backpressure without hard rejection

Frequently Asked Questions

What happens when all connections in a quota-share combo are saturated?

The request still receives a candidate. Saturated connections are de-prioritized to the end of the list, but never removed. This fail-open behavior ensures service continuity even during quota exhaustion, as implemented in filterEligibleBySaturation within [quotaShareStrategy.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/quotaShareStrategy.ts).

How does DRR differ from simple round-robin in OmniRoute?

Simple round-robin gives equal turns to all connections regardless of weight. Deficit Round-Robin accumulates quantums proportional to each connection's configured weight, allowing weighted fair distribution. A connection with weight 2 receives twice the throughput opportunity of weight 1 over extended sequences, tracked via the _drrState Map in the strategy implementation.

Why use Power-of-Two-Choices instead of selecting the single DRR winner?

P2C provides load-awareness without perfect global knowledge. By comparing the top two DRR candidates by in-flight count, OmniRoute avoids directing traffic to a connection that happens to lead in deficit but is currently overloaded. This two-choice sampling mitigates tail latency without requiring centralized load state synchronization.

Can concurrency caps cause requests to fail with no available target?

No. The partitionByConcurrencyCap logic follows the same fail-open principle as bucket gating. Connections at their max_concurrent limit are ordered behind those with headroom, but remain in the candidate list. A request only fails if the combo configuration itself contains zero connections—never due to transient capacity constraints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}