How FreeLLMAPI Infers Provider Quota Pool Keys and Manages Shared Quota Tracking

The FreeLLMAPI provider quota system uses a deterministic two-step inference process to map platforms and models to canonical pool keys, enabling shared quota tracking across multiple API keys within the same account or project.

The tashfeenahmed/freellmapi repository implements a sophisticated quota management layer that treats different API keys belonging to the same "account" or "project" as shared resources. By intelligently inferring quota pool keys from platform and model identifiers, the system consolidates usage data across multiple keys. This article examines the implementation in server/src/services/provider-quota.ts to explain how the FreeLLMAPI provider quota system determines pool identifiers and maintains accurate shared quota state.

How FreeLLMAPI Infers Quota Pool Keys

The system determines the identifier of a quota pool through platform-specific rules and contextual fallback mechanisms.

Platform-Specific Inference Rules

The core logic resides in the inferPoolForPlatform function (lines 20-40 of server/src/services/provider-quota.ts). This function encodes canonical mapping rules that translate a platform identifier and optional model ID into a deterministic pool key string.

• Default pattern: Most providers map to "${platform}::account" (e.g., google::account).
• Specialized pools: Several platforms implement unique pool identifiers:
  • OpenRouter: Free tier models (IDs ending with :free) map to openrouter::free; paid models map to openrouter::account.
  • NVIDIA: Uses the credit-pool identifier nvidia::credit-pool.
  • Mistral: Maps to the experimental pool mistral::experiment-pool.
  • OpenCode: Uses the promotional pool opencode::promo.
  • Google: Specifically maps to google::project rather than the generic account pattern.

When no hard-coded rule applies and a modelId is provided, the function falls back to "${platform}::${modelId}".

// provider-quota.ts – inference logic
function inferPoolForPlatform(platform: Platform, modelId?: string | null): string {
  const normalizedModelId = modelId?.trim() ?? '';
  if (platform === 'openrouter') return normalizedModelId.endsWith(':free')
    ? 'openrouter::free' : 'openrouter::account';
  if (platform === 'google') return 'google::project';
  …
  return normalizedModelId ? `${platform}::${normalizedModelId}`
    : `${platform}::account`;
}

Contextual Fallback for Request Processing

When processing live requests, the system checks for an existing quota pool key in the current quota-observation context (set by runWithQuotaObservationContext). The extractContext function (lines 63-74) implements a priority-based resolution:

1. Use explicitly provided quotaPoolKey if present.
2. Fall back to the context's existing quotaPoolKey.
3. If neither exists, invoke inferPoolForPlatform using the request's platform and model ID.

// provider-quota.ts – context extraction
function extractContext(opts: Pick<QuotaObservationInput, …>) {
  const context = getQuotaObservationContext();
  const platform = opts.platform ?? context?.platform;
  …
  return {
    platform,
    keyId: …,
    providerAccountId: …,
    modelId: …,
    quotaPoolKey: opts.quotaPoolKey ?? context?.quotaPoolKey
      ?? inferPoolForPlatform(platform, opts.modelId ?? context?.modelId),
    endpoint: …
  };
}

Shared Quota Pool Detection and Tracking

Beyond inference, the system must identify which platforms support shared quota semantics to ensure proper tracking behavior.

Identifying Shared Pool Platforms

The isSharedPool helper function (lines 42-44) maintains a definitive whitelist of platforms treated as shared resources. When a platform is in this list, the system knows that different API keys may consume from a common limit.

// provider-quota.ts – shared‑pool helper
function isSharedPool(platform: Platform): boolean {
  return ['openrouter','google','groq','cerebras','sambanova','nvidia',
          'mistral','github','cohere','cloudflare','zhipu','ollama',
          'kilo','pollinations','llm7','huggingface','opencode'].includes(platform);
}

For shared pools where the provider response lacks explicit quota headers, the system records a low-confidence probe observation to indicate active pool usage without specific limit data.

Recording Observations Across Shared Pools

The quota tracking flow implements a four-stage pipeline that consolidates data by pool key:

1. Context Wrapping: Each request executes within runWithQuotaObservationContext, populating the QuotaObservationContext.
2. Response Parsing: parseQuotaObservationsFromResponse extracts quota headers (e.g., x-ratelimit-limit-requests) using HEADER_SPECS definitions.
3. Data Persistence: recordQuotaObservation upserts the canonical view into provider_quota_state and appends a historical row to provider_quota_observations.
4. Cross-Key Consolidation: Because the quotaPoolKey is deterministic per platform/model combination, all observations from different API keys mapping to the same pool are automatically consolidated, providing a unified shared-quota view.

Practical Implementation Examples

The exported inferQuotaPoolKey wrapper provides a clean interface for the inference logic, while recordQuotaObservationsFromResponse handles post-request processing.

import { inferQuotaPoolKey, recordQuotaObservationsFromResponse } from './services/provider-quota.js';

// Example: infer the pool key for an OpenRouter free‑tier model
const poolKey = inferQuotaPoolKey('openrouter', 'gpt-4o-mini:free');
// → 'openrouter::free'
console.log(poolKey);

// Example: record quota data from a fetch response
await fetch('https://api.openrouter.ai/v1/models')
  .then(res => recordQuotaObservationsFromResponse(res, {
    platform: 'openrouter',
    modelId: 'gpt-4o-mini:free',
    keyId: 123,
  }));

Summary

• FreeLLMAPI treats multiple API keys as shared quota pools when they belong to the same account or project scope.
• The inferPoolForPlatform function in server/src/services/provider-quota.ts encodes platform-specific rules (lines 20-40) to generate deterministic pool keys like openrouter::free or google::project.
• Contextual fallback in extractContext (lines 63-74) ensures requests without explicit pool keys still resolve to the correct shared pool.
• The isSharedPool whitelist (lines 42-44) identifies platforms requiring shared-quota semantics.
• Observations persist to provider_quota_state and provider_quota_observations, consolidating data across all keys sharing the same pool identifier.

Frequently Asked Questions

What is a quota pool key in FreeLLMAPI?

A quota pool key is a canonical string identifier (e.g., openrouter::free or google::project) that represents a shared quota limit across multiple API keys. The system uses this key to aggregate usage data and remaining limits for all keys belonging to the same account, project, or special tier.

How does the system handle OpenRouter's free tier models?

OpenRouter free tier models contain IDs ending with the :free suffix. The inferPoolForPlatform function specifically checks for this suffix and maps such models to the openrouter::free pool key, while paid models map to openrouter::account. This separation ensures that free tier usage does not consume paid quota limits.

Which database tables store quota observations?

The system writes to two SQLite tables: provider_quota_state stores the canonical current view of limits and usage for each pool key, while provider_quota_observations appends historical rows for audit trails and trend analysis. Both tables use the quotaPoolKey as a primary consolidation mechanism.

What happens when a provider doesn't return quota headers?

When a provider belongs to the isSharedPool whitelist but returns no quota headers, FreeLLMAPI records a low-confidence probe observation to indicate the pool is actively being used. This maintains awareness of the shared resource even when specific limit data is unavailable from the provider's response.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}