How Per-Key Rate Limiting Works in FreeLLMAPI: RPM, RPD, TPM, and TPD Explained

FreeLLMAPI enforces per-key rate limits using sliding-window counters stored in memory with SQLite persistence, tracking requests and tokens per minute/day across unique combinations of platform, model, and API key.

FreeLLMAPI implements sophisticated per-key rate limiting to manage LLM provider quotas and prevent API key exhaustion. The system tracks four distinct metrics—RPM, RPD, TPM, and TPD—across unique combinations of platform, model, and API key. This architecture, centered in server/src/services/ratelimit.ts, ensures fair usage while maintaining high availability through in-memory caching with persistent SQLite backing.

The Four Limit Types

FreeLLMAPI enforces four independent quotas per API key:

• RPM (Requests Per Minute): Counts individual API calls within a 60-second window
• RPD (Requests Per Day): Counts total daily requests within a 24-hour window
• TPM (Tokens Per Minute): Sums token usage (input + output) within a 60-second window
• TPD (Tokens Per Day): Accumulates total daily token consumption within a 24-hour window

Each limit operates independently—a key might hit its RPM cap while remaining under its TPM limit, allowing fine-grained traffic shaping.

Sliding-Window Architecture

At the core of server/src/services/ratelimit.ts is a sliding-window implementation using an in-memory Map structure. Each window tracks timestamps of recent activity and token consumption:

interface Window {
  timestamps: number[];
  tokenCount: number;
  tokenTimestamps: { ts: number; tokens: number }[];
}

The system generates composite keys that isolate usage by specific combinations. The key format follows platform:modelId:keyId:type, where type is one of rpm, rpd, tpm, or tpd:

// Key format: "platform:modelId:keyId:type"
const windows = new Map<string, Window>();

This granularity ensures that usage of one model does not count against another model's limits for the same provider key.

Persistence and Fallback Strategy

FreeLLMAPI implements a dual-layer storage system for reliability. When SQLite is available, the system reads counters from the rate_limit_usage table via countPersistedRequests and sumPersistedTokens. If the database is unavailable, the system falls back to in-memory sliding windows through memoryRequestCount and memoryTokenCount functions (lines 58-99).

The counting logic (lines 119-136) prioritizes persisted data while maintaining conversation continuity:

// Pseudo-implementation showing the fallback pattern
function requestCount(key, windowMs) {
  const persisted = countPersistedRequests(key, windowMs);
  if (persisted !== null) return persisted;
  return memoryRequestCount(key, windowMs);
}

This design ensures rate limit accuracy survives server restarts while remaining functional during database outages.

Quota Enforcement and Usage Recording

The service exposes four primary functions for enforcing limits (lines 138-176):

• canMakeRequest(platform, modelId, keyId, limits): Validates RPM and RPD quotas before allowing new requests
• canUseTokens(platform, modelId, keyId, tokenCount, limits): Verifies TPM and TPD limits won't be exceeded by the estimated token count

Both functions compare current usage against configured limits, treating null values as "no limit."

After successful LLM calls, the system records consumption via:

• recordRequest(platform, modelId, keyId): Adds timestamps to RPM/RPD windows and persists a database row (lines 44-73)
• recordTokens(platform, modelId, keyId, tokenCount): Adds token timestamps to TPM/TPD windows with database persistence

Provider-Wide Daily Caps and Cooldowns

Beyond per-key limits, FreeLLMAPI supports provider-wide daily caps for accounts with aggregate quotas (such as OpenRouter's free tier). The system reads optional environment variables (PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>) and falls back to DEFAULT_PROVIDER_DAILY_REQUEST_CAPS. The canUseProvider helper (lines 188-200) checks aggregate daily counts across all models for a given key.

Transient failure handling occurs through an exponential cooldown mechanism. When a provider returns HTTP 429, 402, or 403, the offending key/model enters a cooldown state for TRANSIENT_COOLDOWN_MS, escalating via COOLDOWN_DURATIONS. This state persists in the rate_limit_cooldowns table (lines 74-124) to survive restarts.

For providers that don't publish daily quotas, FreeLLMAPI employs a learning heuristic: after NULL_LIMIT_HIT_THRESHOLD 429 errors within an hour, the system treats the key as effectively exhausted and escalates the cooldown (lines 320-340).

Implementation Example

The following pattern demonstrates complete rate limit integration:

import {
  canMakeRequest,
  canUseTokens,
  recordRequest,
  recordTokens,
  getRateLimitStatus,
} from '@/services/ratelimit';

// Limits typically fetched from provider catalog
const limits = {
  rpm: 30,      // max 30 requests per minute
  rpd: 1000,    // max 1000 requests per day
  tpm: 7500,    // max 7500 tokens per minute
  tpd: 150000,  // max 150k tokens per day
};

const platform = 'openrouter';
const modelId = 'gpt-3.5-turbo';
const keyId = 42;

// Pre-flight checks
if (!canMakeRequest(platform, modelId, keyId, limits)) {
  throw new Error('RPM/RPD limit reached');
}

const estimatedTokens = 500;
if (!canUseTokens(platform, modelId, keyId, estimatedTokens, limits)) {
  throw new Error('TPM/TPD limit would be exceeded');
}

// Execute LLM call...

// Post-call recording
recordRequest(platform, modelId, keyId);
recordTokens(platform, modelId, keyId, actualTokenCount);

// Dashboard inspection
const status = getRateLimitStatus(platform, modelId, keyId, limits);
console.log('RPM used/limit:', status.rpm.used, '/', status.rpm.limit);

Summary

• Four independent metrics: RPM, RPD, TPM, and TPD tracked separately per key
• Composite key isolation: Usage scoped to specific platform:modelId:keyId:type combinations in server/src/services/ratelimit.ts
• Dual-layer persistence: Sliding windows backed by SQLite with automatic in-memory fallback
• Automatic cooldown: HTTP 429/402/403 responses trigger escalating backoff periods stored in rate_limit_cooldowns
• Provider-wide support: Environment-variable daily caps supplement per-key limits for account-level quotas

Frequently Asked Questions

What happens to rate limiting when SQLite is unavailable?

FreeLLMAPI falls back to pure in-memory sliding windows. The memoryRequestCount and memoryTokenCount functions maintain window state in a JavaScript Map, ensuring rate limiting continues to function (though without persistence across restarts) during database outages.

How does FreeLLMAPI handle providers with account-wide daily limits?

The system supports provider-wide daily caps through PROVIDER_DAILY_REQUEST_CAP_<PLATFORM> environment variables. The canUseProvider function checks aggregate daily requests across all models for a given key, complementing the per-model rate limiting in server/src/services/ratelimit.ts.

What triggers the automatic cooldown mechanism?

HTTP responses indicating rate limit exhaustion (429 Too Many Requests), payment required (402), or forbidden (403) trigger automatic cooldowns. The system stores these in the rate_limit_cooldowns table with escalating durations (COOLDOWN_DURATIONS), temporarily blocking the specific platform/model/key combination.

How does the system behave when a provider doesn't specify token limits?

FreeLLMAPI uses a learning heuristic defined by NULL_LIMIT_HIT_THRESHOLD. If a key receives multiple 429 errors within an hour without a published limit, the system treats the key as effectively exhausted and applies extended cooldowns to prevent hammering the provider's undefined quota.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}