Configuring Quota-Share Routing for API Key Distribution in OmniRoute

OmniRoute's quota-share routing distributes API requests across multiple provider connections using virtual model names prefixed with qtSd/ and a dedicated load-balancing strategy that combines Deficit Round Robin (DRR) and Power-of-Two-Choices (P2C) algorithms.

OmniRoute enables intelligent traffic distribution through quota-share routing for API key distribution, allowing a single API key to access aggregated capacity from multiple provider connections. This architecture automatically generates hidden combo models that route requests through specialized balancing logic while respecting per-connection concurrency limits. Understanding how to configure quota-share routing for API key distribution ensures optimal resource utilization and failover resilience.

Architecture and Core Components

The quota-share system relies on several interconnected modules that handle virtual model generation, routing strategy registration, and request distribution.

Virtual Model Generation and Naming

When you create a quota pool, OmniRoute automatically mints hidden combo models through the syncQuotaCombos function in src/lib/quota/quotaCombos.ts (lines 29‑61). These combos use standardized virtual model names beginning with qtSd/, constructed via quotaModelName in src/lib/quota/quotaModelNaming.ts. The naming convention follows the pattern qtSd/<group>/<provider>/<modelId>, creating unique identifiers like qtSd/mygroup/openai/gpt-4 that represent aggregated access to multiple provider connections.

Routing Strategy Registration

The system registers the quota-share routing algorithm in src/shared/constants/routingStrategies.ts (lines 27‑31) using the constant QUOTA_SHARE_STRATEGY. This internal value signals the combo router in open-sse/services/combo.ts to dispatch requests to selectQuotaShareTarget, which implements the DRR and P2C load-balancing logic rather than standard round-robin or random selection.

Configuring Quota Pools and Connections

Setting up quota-share routing requires defining pools, associating provider connections, and optionally configuring concurrency limits.

Step 1: Create a Quota Pool

Define a quota pool via the UI or API endpoint /api/quota/pools/[id]. Each pool requires a groupId and an array of connectionIds pointing to existing provider connections:

{
  "name": "my-quota-pool",
  "groupId": "my-group",
  "connectionIds": ["conn-1", "conn-2"]
}

When saved, OmniRoute triggers syncQuotaCombos(poolId) which fetches registered model IDs from the REGISTRY for each connection and upserts hidden combos (lines 70‑96 in quotaCombos.ts). Each combo contains a single model-step for its respective connection and stores the strategy as QUOTA_SHARE_STRATEGY.

Step 2: Enable Per-Connection Concurrency Limits

To prevent overwhelming individual connections, configure the maxConcurrent field on pool connections. This setting, referenced in src/lib/resilience/settings.ts (line 107), enables the quota-share strategy to serialize concurrent requests rather than rejecting excess traffic. Update the pool connection record with:

{
  "maxConcurrent": 10,
  "enableMaxConcurrent": true
}

When enabled, the resilience guard queues requests that exceed the connection's capacity, ensuring stable throughput across the quota pool.

API Key Distribution and Model Filtering

Quota-share routing integrates deeply with OmniRoute's API key system to control which virtual models each key can access.

Filtering Models to Allowed Pools

API keys store a list of permitted quota pool slugs. The endpoint GET /quota/keys/:id/models in src/app/api/quota/keys/[id]/models/route.ts utilizes filterModelsToQuotaPools (exported from quotaCombos.ts, lines 67‑84) to return only virtual qtSd/* models belonging to the key's authorized pools. This ensures clients receive a sanitized catalog containing only the aggregated models they are allowed to invoke.

Enabling Quota-Share for API Keys

In the key editor (located in src/app/(dashboard)/dashboard/costs/quota-share/), administrators enable the Quota-Share toggle and specify allowed pools. Once activated, the key exclusively sees virtual model names like qtSd/mygroup/openai/gpt-4, routing all requests through the quota-share combo rather than direct provider connections.

Maintenance and Cleanup

When pool configurations change, OmniRoute automatically manages combo lifecycle to prevent stale entries.

Removing Stale Combos

The removeQuotaCombosForPool function (lines 45‑63 in quotaCombos.ts) deletes all combos matching a pool's group and provider when connections are removed. A separate prune loop (lines 18‑60) scans for orphaned combos that no longer reference valid pool connections, ensuring the routing table remains clean without manual intervention.

Code Examples

Synchronizing Pools Manually

For migrations or scripts, manually trigger combo generation:

import { syncQuotaCombos } from '@/lib/quota/quotaCombos';

// poolId corresponds to the SQLite quotaPools.id field
await syncQuotaCombos('pool-1234');

Source: quotaCombos.ts#L29-L61

Filtering Model Catalogs for Keys

Implement custom filtering when building API key interfaces:

import { filterModelsToQuotaPools } from '@/lib/quota/quotaCombos';

const allModels = await fetchModelsFromProvider(); // [{id: 'openai/gpt-4'}, ...]
const poolSlugs = ['mygroup']; // Derived from key's allowed quotas
const visibleModels = filterModelsToQuotaPools(allModels, poolSlugs);
// Returns only models matching qtSd/mygroup/...

Source: quotaCombos.ts#L67-L84

Requesting Quota-Share Models

Clients invoke quota-share routing by specifying the virtual model name:

POST /v1/chat/completions
Authorization: Bearer <quota-share-api-key>
Content-Type: application/json

{
  "model": "qtSd/mygroup/openai/gpt-4",
  "messages": [{ "role": "user", "content": "Hello" }]
}

The request routes through the quota-share combo and balances across the pool's connections using DRR and P2C algorithms.

Summary

• Quota-share routing distributes traffic across multiple provider connections using virtual model names prefixed with qtSd/ and a dedicated routing strategy.
• Automatic combo generation occurs via syncQuotaCombos in src/lib/quota/quotaCombos.ts, which creates hidden combos whenever pool configurations change.
• API key integration filters available models through filterModelsToQuotaPools, ensuring keys access only authorized quota pools.
• Concurrency controls in src/lib/resilience/settings.ts optionally limit simultaneous requests per connection, queuing excess traffic rather than rejecting it.
• Cleanup processes automatically remove stale combos when pools are deleted or modified via removeQuotaCombosForPool.

Frequently Asked Questions

How does OmniRoute balance requests across quota pool connections?

OmniRoute employs a hybrid approach combining Deficit Round Robin (DRR) and Power-of-Two-Choices (P2C) algorithms within the selectQuotaShareTarget function. DRR ensures fair bandwidth allocation across connections, while P2C helps avoid overloading specific instances by randomly selecting two candidates and picking the less loaded one.

What happens when a quota pool's connections change?

When you modify a pool's connectionIds array, OmniRoute automatically invokes syncQuotaCombos (lines 70‑96 in quotaCombos.ts). This function mints new virtual combos for added connections and triggers removeQuotaCombosForPool (lines 45‑63) to prune obsolete entries, ensuring the routing table reflects the current pool state without manual synchronization.

Can I limit concurrent requests to specific connections within a quota pool?

Yes. Configure the maxConcurrent field on individual pool connections and set enableMaxConcurrent: true. As noted in src/lib/resilience/settings.ts (line 107), this setting instructs the quota-share strategy to serialize concurrent requests, maintaining a queue rather than immediately rejecting traffic that exceeds the connection's capacity limit.

How do API keys access quota-share models?

API keys must have the Quota-Share toggle enabled and specify allowed pool slugs. The system then filters the global model catalog via filterModelsToQuotaPools in src/lib/quota/quotaCombos.ts, returning only virtual models matching the qtSd/<group>/<provider>/<model> pattern. Clients use these virtual names in requests, which the combo router directs through the quota-share strategy rather than standard routing paths.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}