# How Per-Key Rate Limiting Works in FreeLLMAPI: RPM, RPD, TPM, and TPD Explained

> Learn how FreeLLMAPI uses per-key rate limiting (RPM RPD TPM TPD) with sliding-window counters and SQLite persistence. Track usage per minute and day for your API keys and models.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: deep-dive
- Published: 2026-06-28

---

**FreeLLMAPI enforces per-key rate limits using sliding-window counters stored in memory with SQLite persistence, tracking requests and tokens per minute/day across unique combinations of platform, model, and API key.**

FreeLLMAPI implements sophisticated per-key rate limiting to manage LLM provider quotas and prevent API key exhaustion. The system tracks four distinct metrics—RPM, RPD, TPM, and TPD—across unique combinations of platform, model, and API key. This architecture, centered in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), ensures fair usage while maintaining high availability through in-memory caching with persistent SQLite backing.

## The Four Limit Types

FreeLLMAPI enforces **four independent quotas** per API key:

- **RPM (Requests Per Minute)**: Counts individual API calls within a 60-second window
- **RPD (Requests Per Day)**: Counts total daily requests within a 24-hour window  
- **TPM (Tokens Per Minute)**: Sums token usage (input + output) within a 60-second window
- **TPD (Tokens Per Day)**: Accumulates total daily token consumption within a 24-hour window

Each limit operates independently—a key might hit its RPM cap while remaining under its TPM limit, allowing fine-grained traffic shaping.

## Sliding-Window Architecture

At the core of [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) is a **sliding-window** implementation using an in-memory Map structure. Each window tracks timestamps of recent activity and token consumption:

```typescript
interface Window {
  timestamps: number[];
  tokenCount: number;
  tokenTimestamps: { ts: number; tokens: number }[];
}

```

The system generates **composite keys** that isolate usage by specific combinations. The key format follows `platform:modelId:keyId:type`, where type is one of `rpm`, `rpd`, `tpm`, or `tpd`:

```typescript
// Key format: "platform:modelId:keyId:type"
const windows = new Map<string, Window>();

```

This granularity ensures that usage of one model does not count against another model's limits for the same provider key.

## Persistence and Fallback Strategy

FreeLLMAPI implements a **dual-layer storage system** for reliability. When SQLite is available, the system reads counters from the `rate_limit_usage` table via `countPersistedRequests` and `sumPersistedTokens`. If the database is unavailable, the system falls back to in-memory sliding windows through `memoryRequestCount` and `memoryTokenCount` functions (lines 58-99).

The counting logic (lines 119-136) prioritizes persisted data while maintaining conversation continuity:

```typescript
// Pseudo-implementation showing the fallback pattern
function requestCount(key, windowMs) {
  const persisted = countPersistedRequests(key, windowMs);
  if (persisted !== null) return persisted;
  return memoryRequestCount(key, windowMs);
}

```

This design ensures rate limit accuracy survives server restarts while remaining functional during database outages.

## Quota Enforcement and Usage Recording

The service exposes four primary functions for enforcing limits (lines 138-176):

- **`canMakeRequest(platform, modelId, keyId, limits)`**: Validates RPM and RPD quotas before allowing new requests
- **`canUseTokens(platform, modelId, keyId, tokenCount, limits)`**: Verifies TPM and TPD limits won't be exceeded by the estimated token count

Both functions compare current usage against configured limits, treating `null` values as "no limit."

After successful LLM calls, the system records consumption via:

- **`recordRequest(platform, modelId, keyId)`**: Adds timestamps to RPM/RPD windows and persists a database row (lines 44-73)
- **`recordTokens(platform, modelId, keyId, tokenCount)`**: Adds token timestamps to TPM/TPD windows with database persistence

## Provider-Wide Daily Caps and Cooldowns

Beyond per-key limits, FreeLLMAPI supports **provider-wide daily caps** for accounts with aggregate quotas (such as OpenRouter's free tier). The system reads optional environment variables (`PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>`) and falls back to `DEFAULT_PROVIDER_DAILY_REQUEST_CAPS`. The `canUseProvider` helper (lines 188-200) checks aggregate daily counts across all models for a given key.

**Transient failure handling** occurs through an exponential cooldown mechanism. When a provider returns HTTP 429, 402, or 403, the offending key/model enters a cooldown state for `TRANSIENT_COOLDOWN_MS`, escalating via `COOLDOWN_DURATIONS`. This state persists in the `rate_limit_cooldowns` table (lines 74-124) to survive restarts.

For providers that don't publish daily quotas, FreeLLMAPI employs a **learning heuristic**: after `NULL_LIMIT_HIT_THRESHOLD` 429 errors within an hour, the system treats the key as effectively exhausted and escalates the cooldown (lines 320-340).

## Implementation Example

The following pattern demonstrates complete rate limit integration:

```typescript
import {
  canMakeRequest,
  canUseTokens,
  recordRequest,
  recordTokens,
  getRateLimitStatus,
} from '@/services/ratelimit';

// Limits typically fetched from provider catalog
const limits = {
  rpm: 30,      // max 30 requests per minute
  rpd: 1000,    // max 1000 requests per day
  tpm: 7500,    // max 7500 tokens per minute
  tpd: 150000,  // max 150k tokens per day
};

const platform = 'openrouter';
const modelId = 'gpt-3.5-turbo';
const keyId = 42;

// Pre-flight checks
if (!canMakeRequest(platform, modelId, keyId, limits)) {
  throw new Error('RPM/RPD limit reached');
}

const estimatedTokens = 500;
if (!canUseTokens(platform, modelId, keyId, estimatedTokens, limits)) {
  throw new Error('TPM/TPD limit would be exceeded');
}

// Execute LLM call...

// Post-call recording
recordRequest(platform, modelId, keyId);
recordTokens(platform, modelId, keyId, actualTokenCount);

// Dashboard inspection
const status = getRateLimitStatus(platform, modelId, keyId, limits);
console.log('RPM used/limit:', status.rpm.used, '/', status.rpm.limit);

```

## Summary

- **Four independent metrics**: RPM, RPD, TPM, and TPD tracked separately per key
- **Composite key isolation**: Usage scoped to specific `platform:modelId:keyId:type` combinations in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts)
- **Dual-layer persistence**: Sliding windows backed by SQLite with automatic in-memory fallback
- **Automatic cooldown**: HTTP 429/402/403 responses trigger escalating backoff periods stored in `rate_limit_cooldowns`
- **Provider-wide support**: Environment-variable daily caps supplement per-key limits for account-level quotas

## Frequently Asked Questions

### What happens to rate limiting when SQLite is unavailable?

FreeLLMAPI falls back to pure in-memory sliding windows. The `memoryRequestCount` and `memoryTokenCount` functions maintain window state in a JavaScript Map, ensuring rate limiting continues to function (though without persistence across restarts) during database outages.

### How does FreeLLMAPI handle providers with account-wide daily limits?

The system supports provider-wide daily caps through `PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>` environment variables. The `canUseProvider` function checks aggregate daily requests across all models for a given key, complementing the per-model rate limiting in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts).

### What triggers the automatic cooldown mechanism?

HTTP responses indicating rate limit exhaustion (429 Too Many Requests), payment required (402), or forbidden (403) trigger automatic cooldowns. The system stores these in the `rate_limit_cooldowns` table with escalating durations (`COOLDOWN_DURATIONS`), temporarily blocking the specific platform/model/key combination.

### How does the system behave when a provider doesn't specify token limits?

FreeLLMAPI uses a learning heuristic defined by `NULL_LIMIT_HIT_THRESHOLD`. If a key receives multiple 429 errors within an hour without a published limit, the system treats the key as effectively exhausted and applies extended cooldowns to prevent hammering the provider's undefined quota.