How to Implement Quota-Share Routing in OmniRoute for Fair Distribution of Quotas

Quota-Share routing in OmniRoute dynamically distributes provider quotas across combo targets by calculating fair-share weights based on real-time consumption data, ensuring no single target monopolizes available capacity.

OmniRoute is an open-source routing layer that enables intelligent request distribution across multiple AI providers. When you need to ensure equitable access to limited provider quotas across different targets, Quota-Share routing provides a dynamic distribution mechanism that automatically adjusts traffic allocation based on remaining capacity.

Understanding Quota-Share Routing Architecture

The Quota-Share implementation relies on a transactional SQLite schema that tracks consumption and allocation across three primary dimensions: pools, groups, and combos.

Core Database Schema

The system stores quota metadata in several tables managed through migrations located in src/lib/db/migrations/. The file 107_quota_combos_quota_share_strategy.sql adds the necessary strategy column to quota_combos and creates supporting tables including quota_pools, quota_groups, and quota_consumption.

These tables maintain the relationship between provider allocations and actual usage. When you configure a combo with strategy: "quota_share", OmniRoute references these tables to determine how to distribute incoming requests.

Strategy Registration

The strategy binding occurs in src/lib/quota/quotaCombos.ts, which registers the quota-share policy in the combo-resolution pipeline. When the combo router in open-sse/services/combo.ts encounters a combo configured with strategy: "quota_share", it delegates target selection to the quota subsystem.

Implementing the Quota-Share Calculation Logic

The core algorithm resides in src/lib/quota/fairShare.ts. This module calculates normalized weights for each target using the formula: weight = (remainingQuota / totalQuota) * priorityFactor.

The function reads current consumption from quota_consumption and total allocations from quota_pools, then computes proportional weights that sum to 1. Targets with higher remaining quota receive proportionally larger traffic shares.

To create a combo that uses this strategy:

import { createCombo } from '@/lib/db/combo';
import { ProviderId, ModelId } from '@/shared/constants/providers';

await createCombo({
  name: 'fair-share-gpt-combo',
  strategy: 'quota_share',
  targets: [
    { providerId: ProviderId.OpenAI, modelId: ModelId['gpt-4o'] },
    { providerId: ProviderId.Anthropic, modelId: ModelId['claude-3-5-sonnet'] },
  ],
});

Configuring Quota Pools and Targets

Before the router can distribute traffic fairly, you must define the total available capacity for each provider. The src/lib/quota/planResolver.ts module resolves quota-share plans by reading pool configurations and consumption data.

Set up quota pools using the database helpers:

import { upsertQuotaPool } from '@/lib/db/quotaPools';

await upsertQuotaPool({
  poolId: 'global-openai',
  totalQuota: 1_000_000,
  resetPeriodSec: 86_400,
});

await upsertQuotaPool({
  poolId: 'global-anthropic',
  totalQuota: 500_000,
  resetPeriodSec: 86_400,
});

Each pool defines the total request units available and the reset interval in seconds. The resolver combines this data with current consumption to drive the fair-share calculation.

Routing Flow and Request Lifecycle

When a request arrives, OmniRoute processes it through a five-stage pipeline:

1. API Validation: src/app/api/v1/.../route.ts validates the request and extracts quota-related headers.
2. Strategy Resolution: The combo router checks the combo configuration. If strategy equals "quota_share", it invokes src/lib/quota/quotaCombos.ts.
3. Weight Calculation: src/lib/quota/fairShare.ts computes current weights for all targets based on real-time consumption.
4. Target Selection: The router performs a weighted random selection or deterministic least-used ordering based on the computed weights.
5. Dispatch: The selected target receives the request, subject to final quota verification.

Enforcing Quota Limits at Runtime

The final safeguard occurs in src/lib/quota/enforce.ts. Before dispatching to the upstream provider, this module performs an atomic check against the remaining quota. If the request would exceed the target's allocation, OmniRoute returns a 429 Too Many Requests response. Otherwise, it atomically decrements the quota counter.

For custom middleware or administrative tools, you can invoke enforcement manually:

import { enforceQuota } from '@/lib/quota/enforce';

await enforceQuota({
  comboName: 'fair-share-gpt-combo',
  targetProvider: ProviderId.OpenAI,
  requestUnits: 100,
});

Scaling with Redis for High-Throughput Deployments

By default, OmniRoute stores quota data in SQLite for transactional safety. However, for high-concurrency environments, you can enable src/lib/quota/redisQuotaStore.ts by setting the environment variable QUOTA_STORE=redis.

The Redis-backed store implements the same fair-share algorithm but provides faster concurrent updates, making it suitable for production deployments with thousands of requests per second.

Summary

• Configure quota pools using upsertQuotaPool to define total capacity and reset periods for each provider.
• Create combos with strategy: "quota_share" to activate fair-share distribution across targets.
• The fair-share algorithm in src/lib/quota/fairShare.ts calculates weights based on remaining quota relative to total allocation.
• Runtime enforcement in src/lib/quota/enforce.ts prevents quota overruns by returning HTTP 429 when limits are reached.
• Enable Redis storage via QUOTA_STORE=redis for high-throughput scenarios requiring sub-millisecond quota updates.

Frequently Asked Questions

What is the difference between quota-share and round-robin routing in OmniRoute?

Round-robin distributes requests sequentially without considering capacity limits, while quota-share routing actively weights targets based on their remaining quota allocation. According to the OmniRoute source code in src/lib/quota/fairShare.ts, quota-share prevents quota exhaustion on heavily used targets by automatically reducing their traffic share as consumption approaches the pool limit.

How does OmniRoute handle quota exhaustion when using quota-share routing?

When a target's remaining quota is insufficient for the current request, src/lib/quota/enforce.ts aborts the request with a 429 Too Many Requests status code. The system performs this check atomically to prevent race conditions where multiple simultaneous requests might exceed the quota limit.

Can I use quota-share routing with multiple providers in the same combo?

Yes. The quota-share strategy supports heterogeneous target lists containing different providers and models. The fair-share calculation in src/lib/quota/fairShare.ts treats each target independently, computing weights based on each provider's specific pool consumption, allowing you to balance traffic across OpenAI, Anthropic, and other providers within a single combo.

How do I debug fair-share weight calculations in OmniRoute?

Import the calculateFairShare function from src/lib/quota/fairShare.ts to inspect current weights programmatically. This function returns the normalized weight distribution for all targets in a combo based on real-time consumption data from the quota_consumption table, enabling you to verify that traffic distribution aligns with your quota allocation intentions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}