How to Implement Quota-Share for Distributing Provider Quotas Across Team Keys in OmniRoute

OmniRoute’s Quota-Share engine implements a work-conserving fair-share algorithm that distributes upstream provider limits across team API keys based on configurable weights, enforcing limits before requests reach the upstream executor.

OmniRoute enables multiple API keys to share a single upstream provider account without allowing one key to exhaust the entire quota. The Quota-Share implementation provides a fair-share mechanism that tracks rolling consumption per dimension and enforces allocation policies via the enforceQuotaShare function in src/lib/quota/enforce.ts.

Understanding Quota-Share Architecture

The Quota-Share system centers on quota pools—logical buckets that group API keys under a single provider connection. Each pool manages allocations with specific weights and policies, while the fair-share algorithm decides whether to allow or block requests based on real-time consumption.

Core Concepts

The implementation relies on four key abstractions defined in src/lib/quota/types.ts and src/lib/quota/dimensions.ts:

• Quota Pool: A logical container for a provider connection (e.g., a Codex account) that groups API keys. Persisted in the quota_pools table via src/lib/db/quotaPools.ts.
• Allocation: Each key’s membership in a pool carries a weight (0–100 %) and optional absolute cap. The weight determines the key’s fair-share of the pool’s global limit.
• Dimension: Quotas track consumption across multiple units—percent, requests, tokens, or USD—and time windows (5 h, hourly, daily).
• Saturation Threshold: When global usage exceeds the threshold (default 0.5, configurable via QUOTA_SATURATION_THRESHOLD), the engine switches from generous mode (allows borrowing) to strict mode (enforces fair-share limits).

Database Schema

The SQLite schema defined in src/lib/db/quotaPools.ts stores pools and their allocations:

// src/lib/quota/types.ts
export const QuotaPoolSchema = z.object({
  id: z.string().min(1),
  connectionId: z.string().min(1),
  name: z.string().min(1),
  createdAt: z.string().datetime(),
  allocations: z.array(PoolAllocationSchema).default([]),
});

Each allocation references an apiKeyId and stores its weight, policy (hard, soft, or burst), and optional caps.

Configuring Quota Pools and Allocations

Before enforcement, you must create a pool and assign allocations. The Zod schemas in src/lib/quota/types.ts ensure type safety throughout the pipeline.

Adding a New Allocation

Use the database helpers in src/lib/db/quotaPools.ts to programmatically add a team key to an existing pool:

import { getPool, addAllocationToPool } from "@/lib/db/quotaPools";
import { PoolAllocation } from "@/lib/quota/types";

// Fetch existing pool for provider "codex"
const pool = getPool("pool-codex-01");

const allocation: PoolAllocation = {
  apiKeyId: "team-key-42",
  weight: 20,               // 20% of pool quota
  policy: "hard",           // block when over-share
  capValue: 2000,           // optional absolute cap
  capUnit: "requests",
};

addAllocationToPool(pool.id, allocation);

Resolving Provider Plans

The planResolver.ts module determines which quota dimensions apply to a provider. Resolution follows this precedence:

1. Manual DB override (provider_plans table)
2. Built-in catalog (src/lib/quota/planRegistry.ts)
3. Empty plan (requires manual configuration)

The Fair-Share Algorithm

The core decision logic lives in decideFairShare() within src/lib/quota/fairShare.ts. This function calculates whether a specific API key’s request falls within its fair allocation.

Strict vs. Generous Mode

The algorithm operates in two modes based on global saturation:

• Generous mode (globalUsedPercent < QUOTA_SATURATION_THRESHOLD): Allows keys to borrow unused capacity from other allocations.
• Strict mode: Enforces the calculated fair-share strictly: fairShare = (allocation.weight / 100) * dim.limit.

Enforcement Policies

Each allocation specifies a policy that determines the action when a key exceeds its fair-share:

• Hard: Immediately block the request with a 429 response.
• Soft: Allow the request but mark it as penalized for monitoring.
• Burst: Allow the request if any global headroom remains across the pool.

The function also checks absolute caps before fair-share calculations:

// src/lib/quota/fairShare.ts
if (allocation.capValue !== undefined && 
    allocation.capUnit === dim.key.unit &&
    consumed >= allocation.capValue) {
  return { kind: "block", reason: "cap-absolute" };
}

Enforcing Quotas in the Request Pipeline

Quota enforcement occurs before the request reaches the upstream executor (e.g., chatCore.ts). The enforceQuotaShare function in src/lib/quota/enforce.ts serves as the hot-path hook.

Preflight Check Implementation

Integrate enforcement into your request handler as follows:

import { enforceQuotaShare } from "@/lib/quota/enforce";

async function preflightCheck(req) {
  const decision = await enforceQuotaShare({
    apiKeyId: req.headers["x-omniroute-apikey"],
    connectionId: req.headers["x-omniroute-connection-id"],
    provider: req.body.provider,
    model: req.body.model,          // optional
  });

  if (decision.kind === "block") {
    throw new Response(decision.reason, { 
      status: decision.httpStatus ?? 429 
    });
  }
  // Proceed to upstream executor
}

The EnforceInput type in src/lib/quota/types.ts defines the required parameters.

Consumption Tracking

After receiving an upstream response, record the actual usage via recordConsumption() in the same file. This updates sliding-window counters via SQLite (src/lib/quota/sqliteQuotaStore.ts) or Redis (src/lib/quota/redisQuotaStore.ts). Failures during recording never affect the client response (fail-safe).

Fail-Open Design

Following the “fail-open” principle (B16), any error—such as a missing plan or database unavailability—results in an allow decision, ensuring service continuity.

Storage Drivers and Configuration

OmniRoute supports two storage backends for sliding-window counters:

• SQLite (src/lib/quota/sqliteQuotaStore.ts): Zero-install default, suitable for single-instance deployments.
• Redis (src/lib/quota/redisQuotaStore.ts): Required for multi-node deployments to ensure consistency.

Both drivers implement a two-bucket approximation (current + previous window) for smooth rolling values. Configure via environment variables:

QUOTA_STORE_DRIVER=redis
QUOTA_STORE_REDIS_URL=redis://localhost:6379

Or override via the settings table (quotaStore.driver).

Monitoring and UI Configuration

The Quota-Share dashboard at /dashboard/settings provides a visual interface for pool management. The Quota Pool Wizard (src/app/dashboard/quota-pool-wizard/) assists admins in creating pools and setting per-dimension limits.

Real-time saturation signals are available via src/lib/quota/saturationSignals.ts, and the CLI command (bin/cli/commands/quota.mjs) enables terminal-based pool inspection.

Summary

• Quota Pools group API keys under a provider connection with weighted allocations stored in src/lib/db/quotaPools.ts.
• Fair-Share Algorithm in src/lib/quota/fairShare.ts calculates per-key limits using weights and enforces hard, soft, or burst policies.
• Hot-Path Enforcement via enforceQuotaShare in src/lib/quota/enforce.ts blocks over-quota requests before they reach the provider.
• Storage Drivers support SQLite (default) or Redis (multi-node), configured via QUOTA_STORE_DRIVER.
• Fail-Open Design ensures that database or configuration errors result in allowed requests rather than outages.

Frequently Asked Questions

What is the difference between hard, soft, and burst policies in Quota-Share?

Hard policy blocks requests immediately when a key exceeds its fair-share or absolute cap, returning HTTP 429. Soft policy allows the request but flags it as penalized for monitoring purposes. Burst policy permits requests as long as the pool has unused global headroom, making it ideal for traffic spikes when other keys are idle.

How does OmniRoute handle quota enforcement when the database is unavailable?

The system follows a fail-open principle. If enforceQuotaShare encounters any error—such as a missing quota plan or unreachable storage—it returns an allow decision. This ensures that temporary infrastructure issues do not block legitimate traffic.

What quota storage drivers are available in OmniRoute?

OmniRoute supports SQLite (default, zero-install) via src/lib/quota/sqliteQuotaStore.ts and Redis via src/lib/quota/redisQuotaStore.ts. Redis is recommended for multi-instance deployments to maintain consistent counters across nodes. Set QUOTA_STORE_DRIVER=redis and QUOTA_STORE_REDIS_URL to enable the Redis driver.

How do I set an absolute cap on a specific API key?

When creating a PoolAllocation in src/lib/db/quotaPools.ts, specify the capValue and capUnit fields alongside the weight. For example, set capValue: 2000 and capUnit: "requests" to limit a key to 2000 requests regardless of its fair-share percentage. The decideFairShare function checks this cap before evaluating fair-share limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}