Configuring Key Pools with Fair-Share Quota Distribution in OmniRoute

To configure key pools with fair-share quota distribution in OmniRoute, you create a quota pool grouping multiple API keys from the same provider, assign each key a weight percentage, and let the pool registry distribute quota consumption proportionally across the weighted keys.

OmniRoute implements a sophisticated quota-pool engine that allows operators to share rate limits across multiple provider connections using weighted fair-share algorithms. This architecture is isolated in dedicated services and database layers within the diegosouzapw/OmniRoute repository, enabling fine-grained control over how shared API quotas are consumed across your infrastructure.

Understanding the Quota-Pool Architecture

The quota-sharing logic relies on three primary components: persistent storage for pool definitions, a runtime registry for live session management, and strict validation rules to maintain provider isolation.

Core Database Schema

The persistent layer stores pool metadata and allocation rules in SQLite. The migration file src/lib/db/migrations/085_quota_pools.sql defines two critical tables:

• quota_pools: Stores pool metadata including the primary connection ID and pool name
• quota_allocations: Stores per-key weight allocations as REAL values (0-100%)

These tables work in tandem with src/lib/db/quotaPools.ts, which implements the CRUD operations createPool, updatePool, and deletePool. The database enforces that all connections within a single pool must belong to the same provider, preventing meaningless cross-provider quota mixing.

Pool Registry and Runtime Management

At runtime, the Pool Registry (open-sse/services/sessionPool/poolRegistry.ts) acts as a singleton that holds live SessionPool instances for each provider. This registry offers:

• Registration: PoolRegistry.register(provider, myPool) for executor integration
• Statistics: PoolRegistry.getStats(provider) for health monitoring
• Warm-up: PoolRegistry.warmPool(provider, targetCount) for pre-initializing sessions
• Reset: PoolRegistry.resetPool(provider) for clearing pools after policy changes

The type definitions in open-sse/services/sessionPool/types.ts provide the typed contracts for fingerprints, sessions, pool configuration, and health snapshots, ensuring type safety across the distributed components.

Fair-Share Allocation Algorithm

When a request consumes quota from a pooled resource, OmniRoute distributes the cost proportionally based on assigned weights. The logic follows these steps:

1. Calculate total weight: Sum all active key weights in the pool (stored in quota_allocations.weight)
2. Derive individual share: Compute share = weight / totalWeight for each key
3. Apply consumption: Deduct share * quota_delta from each key's allocation

Because weights are stored as real numbers (floating-point), the system gracefully handles fractional allocations and any number of keys while preserving the overall pool limit. The visual flow is documented in docs/diagrams/pool-fair-share.svg, showing how weighted keys share the quota budget.

Single-Provider Enforcement

To maintain semantic integrity—since different LLM providers operate independent rate limits—OmniRoute strictly enforces single-provider pools. The validation logic in src/lib/db/quotaPools.ts throws an error if you attempt to create or update a pool containing connections from multiple providers.

The test suite in tests/unit/quota-pool-single-provider.test.ts verifies this behavior:

• Creating a pool with mixed providers throws /single provider/i
• Updating a pool to include mixed providers also triggers validation errors
• Same-provider updates succeed without restriction

This constraint ensures that fair-share distribution only occurs among keys that belong to the same LLM provider, preventing logical inconsistencies in quota accounting.

Managing Pools via Code and MCP Tools

OmniRoute exposes pool management through both programmatic APIs and the MCP console tools defined in open-sse/mcp-server/tools/poolTools.ts.

Creating a Fair-Share Pool

First, establish provider connections, then group them into a pool:

import { createPool } from "@/src/lib/db/quotaPools";
import { createProviderConnection } from "@/src/lib/db/providers";

const connA = await createProviderConnection({
  provider: "openai",
  authType: "apikey",
  name: "key-A",
  apiKey: "sk-A…",
});

const connB = await createProviderConnection({
  provider: "openai",
  authType: "apikey",
  name: "key-B",
  apiKey: "sk-B…",
});

const pool = createPool({
  connectionId: connA.id,
  connectionIds: [connA.id, connB.id],
  name: "OpenAI-FairShare",
});

This pattern creates a pool sharing quota between connA and connB. The connectionId field designates the primary connection, while connectionIds lists all participants.

Allocating Weights to API Keys

Assign consumption percentages using the setAllocation function:

import { setAllocation } from "@/src/lib/db/quotaPools";

await setAllocation(pool.id, connA.id, {
  weight: 70,
  policy: "hard",
});

await setAllocation(pool.id, connB.id, {
  weight: 30,
  policy: "soft",
});

The weight parameter accepts values 0-100, representing the percentage of pool quota consumed by that key. The policy field determines enforcement strictness—hard enforces strict limits while soft allows temporary burst over-usage.

Warming Up Sessions

Reduce first-request latency by pre-initializing browser sessions:

import { PoolRegistry } from "@/open-sse/services/sessionPool/poolRegistry";

await PoolRegistry.warmPool("openai", 10);

This creates 10 initialized sessions for the OpenAI provider pool, reusing browser fingerprints and reducing cold-start overhead. The warm-up respects the maxSessions limit defined in PoolConfig.

Monitoring and Resetting Pools

Retrieve runtime statistics to monitor pool health:

const stats = PoolRegistry.getStats("openai");
console.log("Active sessions:", stats.activeCount);
console.log("Success rate:", stats.successRate);

When quota policies change, reset the pool to clear lingering state:

const success = PoolRegistry.resetPool("openai");
if (success) {
  console.log("Pool reset complete");
}

Resetting discards the current SessionPool instance; new requests will trigger fresh pool creation on demand.

Handling Edge Cases and Validation

When configuring key pools with fair-share quota distribution, consider these boundary conditions:

• Zero total weight: If all keys have weight 0, the system treats this as a no-allocation state and rejects requests with a clear error
• Weight normalization: When weights sum to more than 100, the algorithm automatically normalizes by dividing each weight by totalWeight before applying shares
• Session limits: The PoolConfig.maxSessions parameter caps concurrent sessions; warm-up operations respect this ceiling
• Graceful shutdown: When removing connections, call PoolRegistry.unregister(provider) before deleting database rows to ensure in-flight sessions shut down cleanly

Summary

• Quota pools group multiple API keys from a single provider to share a common quota budget, managed in quota_pools and quota_allocations tables
• Fair-share distribution uses proportional weights (0-100%) to divide quota consumption among keys, calculated as weight / totalWeight
• Single-provider enforcement prevents mixed-provider pools through database-level validation in src/lib/db/quotaPools.ts
• Pool Registry (poolRegistry.ts) provides runtime management via singleton pattern, offering registration, warm-up, stats retrieval, and reset capabilities
• MCP tools (poolTools.ts) expose pool operations to operators without requiring code changes
• Weight policies (hard vs soft) control whether keys can burst beyond their allocated shares during traffic spikes

Frequently Asked Questions

How does OmniRoute prevent mixing API keys from different providers in the same pool?

The CRUD operations in src/lib/db/quotaPools.ts validate that all connectionIds belong to the same provider before persisting the pool. If you attempt to create or update a pool with mixed providers (e.g., OpenAI and Anthropic keys together), the system throws a validation error matching /single provider/i. This validation is tested in tests/unit/quota-pool-single-provider.test.ts.

What happens if the sum of all weights in a pool exceeds 100?

The fair-share algorithm normalizes weights by dividing each key's weight by the totalWeight of all active keys in the pool. This means a configuration with weights 70, 50, and 30 (total 150) would result in effective shares of 46.7%, 33.3%, and 20% respectively. The normalization ensures the total quota consumption never exceeds the pool budget while respecting relative priority.

Can I change pool weights without restarting OmniRoute?

Yes. Update the allocation rows in the quota_allocations table using setAllocation() from src/lib/db/quotaPools.ts. The Pool Registry reads these values at request time, so changes take effect immediately for subsequent quota consumption calculations. However, in-flight requests continue using the weight values active at the time of their initiation.

How do I handle a pool when one API key becomes invalid?

Call PoolRegistry.resetPool(provider) to clear the current session pool, then update the database to remove the invalid connection ID from the pool's connectionIds array. Alternatively, set the invalid key's weight to 0 in quota_allocations to effectively remove it from fair-share calculations without deleting the connection record. The next request will trigger a fresh pool initialization excluding the zero-weight key.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}