Quota-Share Routing in OmniRoute: How API-Key Distribution Balances Traffic Across Multiple Keys

Quota-share routing in OmniRoute automatically distributes API requests across multiple keys by checking remaining quota windows before each request, excluding exhausted keys from the routing pool to prevent rate-limit failures.

OmniRoute implements a sophisticated quota-share routing mechanism that treats each API key as an independent connection with its own quota windows. This design ensures high availability when running multiple provider accounts or keys. This article explores how the system works, from pre-flight checks to dashboard visualization, based on the OmniRoute source code.

How Quota-Share Routing Works

The quota-share system operates in three coordinated stages: registration, pre-flight evaluation, and dynamic selection. Each stage is implemented in specific modules that work together to route traffic intelligently.

Quota Fetcher Registration

Every provider must register a function capable of retrieving current usage statistics. The registry resides in open-sse/services/quotaPreflight.ts.

This registration pattern allows OmniRoute to support any provider with a usage endpoint—whether they report minute-by-minute consumption or simple monthly totals. The fetcher returns a standardized QuotaInfo object containing:

• used and total counts
• percentUsed for quick threshold checks
• windows object with time-bucketed usage (monthly, weekly, daily)

// Register a quota fetcher for a new provider (e.g., "myProvider")
import { registerQuotaFetcher } from '@/open-sse/services/quotaPreflight';

registerQuotaFetcher('myProvider', async (connectionId) => {
  const resp = await fetch(`https://api.myprovider.com/usage/${connectionId}`);
  const data = await resp.json();
  return {
    used: data.used,
    total: data.limit,
    percentUsed: data.used / data.limit,
    windows: {
      monthly: { percentUsed: data.monthly.used / data.monthly.limit, resetAt: data.monthly.resetAt },
      weekly:  { percentUsed: data.weekly.used / data.weekly.limit, resetAt: data.weekly.resetAt },
    },
  };
});

Pre-Flight Quota Evaluation

Before any request proceeds, preflightQuota evaluates the connection's health. This function is invoked from src/sse/services/auth.ts::getProviderCredentialsWithQuotaPreflight.

The evaluation compares remaining quota against two configurable thresholds:

Threshold	Default	Action
Blocking	2%	Connection excluded from routing pool
Warning	20%	Logged but still usable

If any quota window falls below its minimum, the connection fails pre-flight and the router tries alternatives.

// Perform a quota pre-flight before picking a connection
import { preflightQuota } from '@/open-sse/services/quotaPreflight';
import { getProviderCredentialsWithQuotaPreflight } from '@/src/sse/services/auth';

async function chooseConnection(provider: string, connId: string) {
  const preflight = await preflightQuota(connId);
  if (!preflight.proceed) {
    console.warn(`Connection ${connId} blocked: ${preflight.reason}`);
    return null; // router will try another key
  }
  return getProviderCredentialsWithQuotaPreflight(provider, connId);
}

Dynamic Key Selection in the Combo Router

The combo router (open-sse/services/combo.ts) builds the final list of viable targets. When quota-share is enabled, it:

1. Filters connections that passed pre-flight
2. Prioritizes keys with healthy remaining quota
3. Automatically falls back when a key nears exhaustion

This eliminates manual key rotation and prevents the cascade failures that occur when a single exhausted key continues receiving traffic.

Dashboard Visualization: Monitoring Quota Pools

Operators need visibility into per-key consumption. The quota-share dashboard renders each API key as a Pool with dimensional usage tracking.

Safe Rendering with PoolCard.tsx

The PoolCard.tsx component (src/app/(dashboard)/dashboard/costs/quota-share/components/PoolCard.tsx) demonstrates defensive coding for potentially missing data:

import type { UsageSnapshot } from '@/lib/types';

export function PoolCard({ usage }: { usage?: UsageSnapshot }) {
  const dimensions = usage?.dimensions ?? [];               // ← guard missing array
  const hasDimensions = !!usage?.dimensions?.length;       // ← guard length access
  
  return (
    <div className="pool-card">
      {hasDimensions ? (
        dimensions.map(d => (
          <span key={d.perKey}>{d.perKey ?? '–'}: {d.percentUsed}%</span>
        ))
      ) : (
        <span>No usage data yet</span>
      )}
    </div>
  );
}

The explicit guards (usage?.dimensions ?? [] and !!usage?.dimensions?.length) ensure the UI renders gracefully even when quota fetchers return incomplete data or when new pools have no historical usage.

Aggregated Usage Tracking

The usePoolsUsageAggregate.ts hook aggregates snapshots across all pools, applying the same null-safety patterns. This gives operators a unified view of consumption patterns without exposing implementation fragility.

Key Files in the Quota-Share System

File	Responsibility
open-sse/services/quotaPreflight.ts	Core registry, fetcher storage, pre-flight logic with threshold handling
open-sse/services/quotaMonitor.ts	Caches quota checks and tracks ongoing evaluations for router performance
src/sse/services/auth.ts (getProviderCredentialsWithQuotaPreflight)	Bridges authentication layer with quota validation
open-sse/services/combo.ts	Implements quota-aware routing strategy in the combo router
src/app/(dashboard)/dashboard/costs/quota-share/components/PoolCard.tsx	Renders individual pool usage with defensive null handling
src/app/(dashboard)/dashboard/costs/quota-share/hooks/usePoolsUsageAggregate.ts	Aggregates multi-pool usage for dashboard overview

Summary

• Quota-share routing distributes API traffic across multiple keys by evaluating remaining quota before each request
• The quotaPreflight.ts registry enables pluggable provider support through fetcher registration
• Pre-flight checks at default 2% blocking and 20% warning thresholds automatically exclude exhausted keys
• The combo router performs dynamic key selection with automatic fallback to healthy alternatives
• Dashboard components use defensive rendering patterns (?? [], ?.length checks) to handle missing quota data gracefully
• PoolCard.tsx and usePoolsUsageAggregate.ts provide operator visibility into per-key consumption windows

Frequently Asked Questions

What happens if a quota fetch fails during pre-flight?

If the fetcher throws or returns malformed data, the pre-flight can be configured to either fail open (allow routing with degraded confidence) or fail closed (exclude the connection). The quotaMonitor.ts cache provides stale-but-usable data briefly to prevent cascading failures across multiple keys.

Can different providers use different quota window configurations?

Yes. Each registered fetcher in quotaPreflight.ts defines its own window structure—monthly, weekly, daily, or even custom intervals like "per-minute" for high-frequency providers. The threshold evaluation iterates over whatever windows exist in the returned QuotaInfo.

How does the combo router prioritize between multiple healthy keys?

When multiple connections pass pre-flight, the combo router applies additional strategies (round-robin, weighted random, or latency-based) after the quota filter. Quota-share acts as a gate, not the final selection algorithm—ensuring only viable candidates reach the downstream routing logic.

Is quota-share routing compatible with single-key deployments?

Yes. The mechanism degrades gracefully: with only one key registered, pre-flight either permits or blocks all traffic. Operators see the same PoolCard UI with a single pool, and the blocking threshold still prevents hard rate-limit errors by rejecting requests proactively rather than allowing provider-side rejection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}