How FreeLLMAPI Tracks RPM, RPD, TPM, and TPD Limits for Rate Limiting

FreeLLMAPI enforces rate limits by combining in-memory sliding windows with SQLite persistence, using composite keys to track requests per minute (RPM), requests per day (RPD), tokens per minute (TPM), and tokens per day (TPD) across both volatile and durable storage layers.

FreeLLMAPI is an open-source LLM proxy that implements sophisticated quota management to prevent API key exhaustion. The system tracks four distinct dimensions of usage—RPM, RPD, TPM, and TPD—through a hybrid architecture defined in server/src/services/ratelimit.ts that balances high-performance real-time checks with durable historical record-keeping.

Composite Key Architecture for Rate Limit Windows

The Window Key Structure

Every rate limit check is isolated using a composite key that combines the platform identifier, model ID, API key ID, and limit type. The system generates keys following the pattern:

platform:modelId:keyId:type

Here, type is one of rpm, rpd, tpm, or tpd (as implemented in lines 11-13 of server/src/services/ratelimit.ts). This ensures that usage tracked for one model does not interfere with limits for another model or API key.

In-Memory Sliding Windows

The system maintains a Map<string, Window> that stores live usage data for fast access. The Window interface contains two distinct arrays:

• timestamps: An array of request timestamps used for calculating RPM and RPD
• tokenTimestamps: An array of {ts, tokens} objects used for calculating TPM and TPD (lines 5-9)

When a new request arrives, the system pushes the current timestamp into the appropriate array, creating a sliding window that can be pruned to calculate current usage within the relevant time boundaries.

SQLite Persistence Layer

To ensure rate limits survive server restarts, FreeLLMAPI writes all usage to a SQLite table defined in server/src/db/migrations.ts (lines 100-103). The rate_limit_usage schema stores:

• platform and model_id for resource identification
• key_id for API key tracking
• kind (either 'request' or 'tokens') to distinguish between request counts and token volumes
• tokens and created_at_ms for precise temporal accounting

This dual-layer approach means the system can rebuild its in-memory state from the database after a restart, preventing limit resets during deployments.

Recording Usage in Real-Time

Every processed request triggers two recording functions:

1. recordRequest adds the timestamp to the in-memory window and inserts a row into SQLite via recordUsage (lines 44-53)
2. recordTokens performs the same dual-write for token consumption (lines 56-62)

import { recordRequest, recordTokens } from './services/ratelimit';

// Log a request for key ID 42 using openrouter/gpt-4o-mini
recordRequest('openrouter', 'gpt-4o-mini', 42);

// Log 350 tokens consumed by that request
recordTokens('openrouter', 'gpt-4o-mini', 42, 350);

Querying Limits and Enforcement

Dual-Source Counting Strategy

The system implements a resilient counting mechanism that prioritizes database accuracy while maintaining availability:

• countPersistedRequests and sumPersistedTokens query the SQLite rate_limit_usage table for the specified window (MINUTE for RPM/TPM, DAY for RPD/TPD), returning undefined if the database is unreachable (lines 58-76 and 78-96)
• memoryRequestCount and memoryTokenCount prune entries older than the window duration and count the remaining items, serving as fallbacks when persisted counts are unavailable (lines 100-110)

The unified helpers requestCount and tokenCount automatically prefer persisted values, falling back to in-memory calculations only when necessary (lines 18-23).

Enforcement Decision Points

The actual limit validation occurs in two specialized functions:

• canMakeRequest validates against RPM and RPD limits using requestCount (lines 38-53)
• canUseTokens validates against TPM and TPD limits using tokenCount (lines 56-74)

import { canMakeRequest, canUseTokens } from './services/ratelimit';

const limits = { rpm: 60, rpd: 1000, tpm: 8000, tpd: 100000 };

// Check both request frequency and token budget before forwarding
if (canMakeRequest('openrouter', 'gpt-4o-mini', 42, limits) &&
    canUseTokens('openrouter', 'gpt-4o-mini', 42, estimatedTokens, limits)) {
  // Safe to forward to the LLM provider
}

Provider-Wide Daily Caps

Beyond individual model limits, FreeLLMAPI supports provider-wide daily request quotas that aggregate usage across all models for a specific platform. The system reads the optional environment variable PROVIDER_DAILY_REQUEST_CAP_<PLATFORM> (e.g., PROVIDER_DAILY_REQUEST_CAP_OPENROUTER), defaulting to 1000 requests per day for OpenRouter if unspecified (lines 87-92).

The functions getProviderDailyRequestCap, providerDailyRequestCount, and canUseProvider enforce these global caps (lines 93-102 and 121-132), preventing scenarios where rotating through multiple models could circumvent daily limits.

Monitoring Usage with Status Reports

The getRateLimitStatus function provides real-time visibility into current consumption, returning used counts alongside configured limits for the current minute and day (lines 23-31). This enables dashboard integrations that display remaining quota to users.

import { getRateLimitStatus } from './services/ratelimit';

const status = getRateLimitStatus('openrouter', 'gpt-4o-mini', 42, limits);
console.log(`RPM: ${status.rpm.used}/${status.rpm.limit}`);
console.log(`TPD: ${status.tpd.used}/${status.tpd.limit}`);

Summary

• Composite keys (platform:modelId:keyId:type) isolate rate limit windows per model and API key
• Dual-layer storage combines fast in-memory Map objects with durable SQLite records in rate_limit_usage
• Automatic fallback to memory counters occurs when SQLite queries return undefined, ensuring continuous enforcement
• Dedicated functions canMakeRequest and canUseTokens handle RPM/RPD and TPM/TPD validation respectively
• Provider-wide caps are configurable via PROVIDER_DAILY_REQUEST_CAP_<PLATFORM> environment variables

Frequently Asked Questions

What happens if the SQLite database becomes unavailable?

When countPersistedRequests or sumPersistedTokens encounters a database error and returns undefined, the system automatically falls back to memoryRequestCount and memoryTokenCount. These functions prune expired entries from the in-memory window arrays and calculate usage from the remaining data, ensuring rate limiting continues to function even during database outages.

How does FreeLLMAPI distinguish between request counts and token counts?

The system uses separate storage mechanisms for each metric type. For RPM and RPD, it tracks simple timestamps in the timestamps array. For TPM and TPD, it uses the tokenTimestamps array containing objects with both timestamp and token count properties. The SQLite persistence layer further distinguishes these via the kind column, which stores either 'request' or 'tokens' depending on the usage type being recorded.

Can I configure provider-wide limits instead of per-model limits?

Yes. Set the environment variable PROVIDER_DAILY_REQUEST_CAP_<PLATFORM> where <PLATFORM> is the uppercase provider name (e.g., PROVIDER_DAILY_REQUEST_CAP_OPENROUTER). If unspecified, OpenRouter defaults to 1000 requests per day. The functions getProviderDailyRequestCap and canUseProvider enforce these caps across all models for that provider, preventing aggregate quota exhaustion.

Where are the rate limit windows actually defined in the code?

The core implementation resides in server/src/services/ratelimit.ts, which defines the Window interface, composite key generation, and enforcement logic. The database schema for the rate_limit_usage table is defined in server/src/db/migrations.ts (lines 100-103), which creates the necessary indexes for efficient time-window queries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}