How FreeLLMAPI Tracks RPM, RPD, TPM, and TPD for Rate Limiting

FreeLLMAPI enforces rate limits using a hybrid architecture that combines SQLite persistence for durability with in-memory sliding windows for low-latency lookups, storing usage data under composite keys formatted as platform:modelId:keyId:type.

The open-source FreeLLMAPI repository implements granular quota management by tracking four distinct metrics: Requests Per Minute (RPM), Requests Per Day (RPD), Tokens Per Minute (TPM), and Tokens Per Day (TPD). This tracking system relies on a sophisticated dual-layer storage approach defined in server/src/services/ratelimit.ts, backed by the SQLite schema in server/src/db/migrations.ts.

Composite Key Structure for Rate Limit Windows

Every request is tagged with a composite key that encodes the resource scope and limit type. The system generates keys following the pattern:

platform:modelId:keyId:type

Here, type is one of four literals: rpm, rpd, tpm, or tpd, as implemented in server/src/services/ratelimit.ts lines 11-13. This key structure allows the same underlying storage mechanism to handle both time-based request counting and token summation across different granularities.

Dual-Layer Storage Architecture

FreeLLMAPI maintains usage data in two locations simultaneously to ensure both speed and durability.

SQLite Persistence Layer

All usage events are written to the rate_limit_usage table, which survives server restarts. The schema defined in server/src/db/migrations.ts lines 100-103 creates the following structure:

CREATE TABLE IF NOT EXISTS rate_limit_usage (
    platform TEXT,
    model_id TEXT,
    key_id INTEGER,
    kind TEXT,               -- 'request' or 'tokens'
    tokens INTEGER,
    created_at_ms INTEGER
);

This table stores the raw event stream that powers historical aggregation queries for persisted counters.

In-Memory Sliding Windows

For fast path lookups, the service maintains a Map<string, Window> where each entry corresponds to a composite key. According to the source in server/src/services/ratelimit.ts lines 5-9, each Window object contains:

• timestamps: An array of request timestamps (for RPM/RPD calculations)
• tokenTimestamps: An array of {ts, tokens} objects (for TPM/TPD calculations)

These arrays function as sliding windows that are pruned on each query to remove entries outside the current time boundary.

Recording Usage Events

Every successful request triggers dual writes to both storage layers. The recordRequest function pushes the current timestamp into the in-memory window while recordUsage executes an INSERT into SQLite, as shown in server/src/services/ratelimit.ts lines 44-53. Token consumption follows an identical pattern via recordTokens (lines 56-62).

import { recordRequest, recordTokens } from './services/ratelimit';

// Log a request for key ID 42
recordRequest('openrouter', 'gpt-4o-mini', 42);

// Log 350 tokens consumed by that request
recordTokens('openrouter', 'gpt-4o-mini', 42, 350);

Querying Usage Counts

The system provides unified counting functions that prioritize persisted data but fall back to memory when the database is unavailable.

Persisted Counter Queries

Functions countPersistedRequests and sumPersistedTokens query the SQLite table using time-window filters (MINUTE or DAY). As implemented in server/src/services/ratelimit.ts lines 58-76 and 78-96, these return undefined if the database connection fails, triggering the fallback mechanism.

In-Memory Fallback Counting

When persisted counters return undefined, the system invokes memoryRequestCount or memoryTokenCount. These functions prune expired entries from the timestamps or tokenTimestamps arrays and return the remaining count, as shown in lines 100-110 of server/src/services/ratelimit.ts.

The helper functions requestCount and tokenCount orchestrate this fallback logic automatically, first attempting the persisted query and defaulting to memory counts if needed, as defined in lines 18-23.

Enforcing Limits with Graceful Degradation

Limit checking occurs through two primary gatekeepers. The canMakeRequest function checks RPM and RPD limits using requestCount, while canUseTokens validates TPM and TPD constraints using tokenCount. Both functions respect the window type derived from windowMs (where MINUTE maps to rpm/tpm and DAY maps to rpd/tpd).

import { canMakeRequest, canUseTokens } from './services/ratelimit';

const limits = { rpm: 60, rpd: 1000, tpm: 8000, tpd: 100000 };
const estimatedTokens = 500;

if (
  canMakeRequest('openrouter', 'gpt-4o-mini', 42, limits) &&
  canUseTokens('openrouter', 'gpt-4o-mini', 42, estimatedTokens, limits)
) {
  // Proceed with the API call
}

Provider-Wide Daily Caps

Beyond per-model limits, FreeLLMAPI supports provider-level daily quotas that apply across all models from a single provider. The system reads an optional environment variable PROVIDER_DAILY_REQUEST_CAP_<PLATFORM> and defaults to hard-coded values (e.g., openrouter: 1000), as implemented in server/src/services/ratelimit.ts lines 87-92. Functions getProviderDailyRequestCap, providerDailyRequestCount, and canUseProvider enforce these caps globally.

Exposing Rate Limit Status

For dashboard visibility, getRateLimitStatus returns the current used counts alongside configured limits for both minute and day windows. This function aggregates data from both storage layers to provide real-time quota consumption metrics, as shown in lines 23-31.

import { getRateLimitStatus } from './services/ratelimit';

const status = getRateLimitStatus('openrouter', 'gpt-4o-mini', 42, limits);
console.log(`RPM: ${status.rpm.used}/${status.rpm.limit}`);
console.log(`TPD: ${status.tpd.used}/${status.tpd.limit}`);

Summary

• FreeLLMAPI uses composite keys (platform:modelId:keyId:type) to isolate RPM, RPD, TPM, and TPD windows per credential.
• Dual-layer storage combines SQLite persistence (rate_limit_usage table) with in-memory Map objects for speed and durability.
• Graceful degradation allows rate limiting to continue on memory data alone if the database becomes unavailable.
• Provider-wide caps offer an additional daily request limit across all models from a single provider via environment variables.
• Real-time status reporting exposes current usage versus limits through the getRateLimitStatus function.

Frequently Asked Questions

What happens if the SQLite database becomes unavailable?

If the SQLite database is unreachable, the persisted counter functions return undefined, and the system automatically falls back to in-memory counts stored in the Window objects. This ensures continuous rate limiting even during database outages, though usage data will not survive a server restart until the database recovers.

How does FreeLLMAPI distinguish between request counts and token counts?

Request counts use simple timestamp arrays (timestamps), while token counts use weighted entries (tokenTimestamps) that store both the timestamp and the token quantity. This allows the sliding window to sum actual token consumption rather than just counting requests when enforcing TPM and TPD limits.

What is the difference between per-model and provider-wide rate limits?

Per-model limits (RPM/RPD/TPM/TPD) apply to individual model IDs and are checked via canMakeRequest and canUseTokens. Provider-wide caps apply across all models from a single provider (e.g., all OpenRouter models) and are enforced separately via canUseProvider using the PROVIDER_DAILY_REQUEST_CAP_<PLATFORM> environment variable.

How are the time windows (minute vs. day) determined programmatically?

The system derives the window type from the windowMs parameter passed to counting functions. A window of 60,000ms maps to MINUTE (triggering rpm/tpm checks), while 86,400,000ms maps to DAY (triggering rpd/tpd checks), as implemented in the unified requestCount and tokenCount helpers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}