How to Configure Quota-Share Routing for Distributing Load Across Multiple API Keys in OmniRoute

Quota-share routing in OmniRoute distributes API requests across multiple API keys by creating managed pools with defined allocation percentages, automatically respecting per-key quota limits and concurrency caps.

OmniRoute provides a built-in quota-share routing strategy that enables teams to distribute traffic across multiple API keys belonging to the same provider. This approach prevents individual key exhaustion while optimizing throughput for high-volume AI workloads. By configuring quota-share pools through the dashboard or REST API, you can ensure high availability without manual load balancing.

Understanding Quota-Share Routing Architecture

The quota-share strategy operates through a combo-based routing system defined in src/lib/db/combo.ts. When you save a pool configuration, the server invokes syncQuotaCombos to generate combo entries with strategy: "quota-share" for each connection or model-connection pair. The routing engine in open-sse/services/combo.ts then resolves these combos into a sequence of connection steps, checking quota usage and concurrency limits before selecting a target.

The INTERNAL_ROUTING_STRATEGY_VALUES constant registers the quota-share strategy, with validation coverage in tests/unit/quota-share-strategy.test.ts.

Creating a Quota-Share Pool via the Dashboard

Defining Pools with PoolWizard

The PoolWizard component in src/app/(dashboard)/dashboard/costs/quota-share/components/PoolWizard.tsx provides the interface for grouping connections into quota-share pools. Each pool maps to a specific provider and contains connection IDs representing distinct API keys.

Configure the following parameters for each connection:

• connectionId: The unique identifier for the API key record
• allocation: Percentage of traffic directed to this key (e.g., 60 for 60%)
• maxConcurrent: Optional limit on simultaneous requests per key

Monitoring Pool Health with PoolCard

Once created, the PoolCard component in src/app/(dashboard)/dashboard/costs/quota-share/components/PoolCard.tsx visualizes real-time usage, remaining quota percentages, and per-key allocation health. This component renders the allocation bar and burn rate charts, allowing operators to detect exhausted keys before they impact traffic.

Implementing Quota-Share Pools via REST API

For programmatic configuration, create pools by posting to the OmniRoute API:

// POST /api/settings/quota-share/pools
await fetch('/api/settings/quota-share/pools', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    name: 'OpenAI-Production-Pool',
    provider: 'openai',
    connections: [
      { connectionId: 'ck_prod_1', allocation: 60, maxConcurrent: 5 },
      { connectionId: 'ck_prod_2', allocation: 40, maxConcurrent: 3 },
    ],
  }),
});

The payload persists to the quota_pools table managed by src/lib/db/quotaPool.ts. Upon successful creation, the server automatically generates the internal quota-share combos via syncQuotaCombos.

Managing Pools with React Hooks

Integrate pool management into your dashboard using the usePools hook:

import { usePool } from '@/app/(dashboard)/dashboard/costs/quota-share/hooks/usePools';
import { useUpdatePool } from '@/app/(dashboard)/dashboard/costs/quota-share/hooks/useUpdatePool';

function EditPool({ poolId }: { poolId: string }) {
  const { pool } = usePool(poolId);
  const updatePool = useUpdatePool();

  const onSave = (updated: typeof pool) => {
    updatePool.mutate(updated);
  };

  return (
    <PoolWizard
      initialPool={pool}
      onSubmit={onSave}
    />
  );
}

This pattern leverages the usePools hook defined in src/app/(dashboard)/dashboard/costs/quota-share/hooks/usePools.ts to fetch metadata and mutate pool configurations.

How the Routing Engine Distributes Traffic

When a request arrives, the combo engine executes the following logic:

1. Resolution: The resolveComboTargets function expands the pool into ResolvedComboTarget objects based on the quota-share strategy.
2. Quota Validation: The engine checks current usage against limits in src/lib/db/usage.ts.
3. Concurrency Checks: Active connection counts are verified against maxConcurrent values stored in src/lib/db/connection.ts.
4. Failover: If a connection exceeds quota, the engine applies a short cooldown and attempts the next connection in the pool sequence.

Summary

• Quota-share routing distributes traffic across multiple API keys using allocation percentages defined in pools.
• Create pools via the PoolWizard UI or REST API at /api/settings/quota-share/pools.
• The syncQuotaCombos function in src/lib/db/combo.ts automatically generates combo entries with strategy quota-share.
• The routing engine in open-sse/services/combo.ts respects per-key maxConcurrent limits and quota exhaustion states.
• Monitor pool health through the PoolCard component and usePools hook.

Frequently Asked Questions

What happens when one API key in a quota-share pool reaches its quota?

The OmniRoute combo engine detects quota exhaustion through src/lib/db/usage.ts and automatically routes the request to the next available connection in the pool. If all connections are exhausted, the system returns a quota-exhausted error. The engine applies a short cooldown period before retrying exhausted connections to prevent hammering rate-limited keys.

Can I mix different providers within the same quota-share pool?

No. Each quota-share pool must contain connections belonging to the same provider, as defined in the provider field when creating the pool. The syncQuotaCombos function groups connections by provider to ensure compatibility in the routing strategy. Create separate pools for different providers and configure route selection rules to choose between them.

How does OmniRoute handle concurrency limits across multiple keys?

The combo engine in open-sse/services/combo.ts tracks active requests against each connection's maxConcurrent value. When a connection reaches its limit, the engine serializes additional requests by queuing them or falling back to the next available key in the pool. This prevents individual keys from being overwhelmed while maintaining overall throughput.

Where is the quota-share strategy registered in the codebase?

The strategy is registered in the INTERNAL_ROUTING_STRATEGY_VALUES constant, which enumerates all non-UI routing strategies. The registration is validated by unit tests in tests/unit/quota-share-strategy.test.ts, ensuring the quota-share identifier is recognized by the combo resolution system in src/lib/db/combo.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}