How OmniRoute's Quota-Share Routing Strategy Distributes Time-Based Quotas

OmniRoute's quota-share routing strategy automatically partitions a provider's time-based quota evenly across all connections in a pool by minting internal qtSd/... combos and applying per-connection allowances calculated in selectQuotaShareTarget.

OmniRoute implements an intelligent load distribution mechanism for managing provider rate limits through its quota-share routing strategy. This internal strategy, defined in src/shared/constants/routingStrategies.ts, automatically balances traffic across connection pools to maximize throughput without exceeding time-based quotas. Understanding how this distribution works is essential for configuring high-performance routing in the diegosouzapw/OmniRoute repository.

How Quota-Share Distribution Works

The quota-share strategy operates by splitting time-based windows across multiple connections. When enabled, the system generates auto-minted combos with the qtSd/... prefix that represent the entire pool.

The distribution follows a six-step algorithm implemented in src/lib/quota/quotaCombos.ts:

1. Pool Collection – The system gathers all connections sharing the same provider ID, model ID, and quota-share pool ID.
2. Window Reading – It reads the quota_window_ms field from each connection's quota record to determine the time window (per-minute or per-hour).
3. Allowance Calculation – The remaining pool quota (pool.quotaRemaining) is divided by the number of active connections (pool.connections.length).
4. Target Assignment – Each connection receives a ResolvedComboTarget with a maxTokens or maxRequests value equal to the per-connection allowance.
5. Routing Selection – Requests route to the first target with sufficient remaining quota.
6. Cooldown Handling – If a 403 quota_exhausted error occurs, the combo-cooldown-wait setting in src/lib/resilience/settings.ts triggers a backoff retry.

Core Implementation Files

Routing Strategy Definition

In src/shared/constants/routingStrategies.ts, the quota-share constant defines the internal strategy identifier. This strategy remains internal-only and does not appear in UI routing strategy lists, as it is automatically applied to pools with the quota-share flag enabled.

Quota Distribution Logic

The selectQuotaShareTarget function in src/lib/quota/quotaCombos.ts implements the core distribution algorithm. It creates step-wise routing paths and calculates per-connection allowances by dividing the total remaining quota evenly across the pool's connection count.

Resilience and Concurrency Controls

File src/lib/resilience/settings.ts contains two critical safety mechanisms:

• Combo Cooldown Wait – Configures backoff behavior for transient quota exhaustion errors
• Concurrency Kill-Switch – Optional per-connection "max concurrent" cap that serializes requests to prevent quota oversubscription

Practical Usage Examples

Creating a Quota-Share Pool via CLI

omniroute pool create \
  --provider openai \
  --model gpt-4o \
  --quota-share \
  --connections connA connB connC

The CLI writes the quota_share: true flag into each connection record, triggering OmniRoute to auto-generate qtSd/... combos for the provider/model pair.

Routing Requests via API

POST /api/v1/chat/completions
Authorization: Bearer <API_KEY>
Content-Type: application/json

{
  "model": "qtSd/openai/gpt-4o",
  "messages": [{ "role": "user", "content": "Explain quota-share." }]
}

OmniRoute resolves the combo, distributes remaining quota among connections, and routes to the first connection with sufficient allowance.

Monitoring Pool Usage in the Dashboard

import { usePoolUsage } from '@/app/(dashboard)/dashboard/costs/quota-share/hooks/usePoolUsage';

export function QuotaPoolStatus() {
  const { pools } = usePoolUsage();

  return (
    <ul>
      {pools.map(p => (
        <li key={p.id}>
          {p.name}: {p.remainingQuota} tokens left (distributed over {p.connectionCount} connections)
        </li>
      ))}
    </ul>
  );
}

The hook aggregates per-connection quotas from the pool for display in the Pool Wizard UI components located in src/app/(dashboard)/dashboard/costs/quota-share/components/PoolWizard.tsx.

Summary

• OmniRoute's quota-share strategy automatically distributes time-based quotas across connection pools using internal qtSd/... combo prefixes.
• The selectQuotaShareTarget function in src/lib/quota/quotaCombos.ts calculates even per-connection allowances by dividing remaining pool quota by active connection count.
• Resilience mechanisms in src/lib/resilience/settings.ts handle transient quota exhaustion through cooldown waits and optional concurrency caps.
• The strategy is internal-only, activated automatically for pools with quota-share flags, and monitored via the usePoolUsage React hook.

Frequently Asked Questions

How does OmniRoute calculate per-connection quotas in a quota-share pool?

OmniRoute divides the pool's total remaining quota (pool.quotaRemaining) by the number of active connections (pool.connections.length) in the selectQuotaShareTarget function. Each connection receives an equal allowance of tokens or requests for the current time window.

What happens when a connection in a quota-share pool exhausts its quota?

If a connection returns a 403 quota_exhausted error, the combo-cooldown-wait setting triggers an automatic backoff retry. The request will then route to the next available connection in the pool that still has remaining quota allowance.

Can I disable concurrency limits for quota-share connections?

The concurrency limit is controlled by a kill-switch in src/lib/resilience/settings.ts. When enabled, this feature serializes concurrent requests per connection to prevent quota oversubscription. Disabling it removes this serialization but increases the risk of exceeding rate limits.

Why can't I see quota-share as a routing strategy in the OmniRoute UI?

The quota-share strategy is internal-only and intentionally excluded from UI routing strategy lists. It activates automatically when you create a pool with the quota-share flag enabled, generating the qtSd/... combo prefixes behind the scenes without manual strategy selection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}