# How FreeLLMAPI Tracks RPM, RPD, TPM, and TPD Limits for Rate Limiting

> Discover how FreeLLMAPI tracks RPM, RPD, TPM, and TPD limits using in-memory sliding windows and SQLite persistence for effective request management.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-07-01

---

**FreeLLMAPI enforces rate limits by combining in-memory sliding windows with SQLite persistence, using composite keys to track requests per minute (RPM), requests per day (RPD), tokens per minute (TPM), and tokens per day (TPD) across both volatile and durable storage layers.**

FreeLLMAPI is an open-source LLM proxy that implements sophisticated quota management to prevent API key exhaustion. The system tracks four distinct dimensions of usage—RPM, RPD, TPM, and TPD—through a hybrid architecture defined in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) that balances high-performance real-time checks with durable historical record-keeping.

## Composite Key Architecture for Rate Limit Windows

### The Window Key Structure

Every rate limit check is isolated using a composite key that combines the platform identifier, model ID, API key ID, and limit type. The system generates keys following the pattern:

```

platform:modelId:keyId:type

```

Here, `type` is one of `rpm`, `rpd`, `tpm`, or `tpd` (as implemented in lines 11-13 of [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts)). This ensures that usage tracked for one model does not interfere with limits for another model or API key.

### In-Memory Sliding Windows

The system maintains a `Map<string, Window>` that stores live usage data for fast access. The **Window** interface contains two distinct arrays:

- **`timestamps`**: An array of request timestamps used for calculating RPM and RPD
- **`tokenTimestamps`**: An array of `{ts, tokens}` objects used for calculating TPM and TPD (lines 5-9)

When a new request arrives, the system pushes the current timestamp into the appropriate array, creating a sliding window that can be pruned to calculate current usage within the relevant time boundaries.

## SQLite Persistence Layer

To ensure rate limits survive server restarts, FreeLLMAPI writes all usage to a SQLite table defined in [`server/src/db/migrations.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/migrations.ts) (lines 100-103). The `rate_limit_usage` schema stores:

- `platform` and `model_id` for resource identification
- `key_id` for API key tracking
- `kind` (either 'request' or 'tokens') to distinguish between request counts and token volumes
- `tokens` and `created_at_ms` for precise temporal accounting

This dual-layer approach means the system can rebuild its in-memory state from the database after a restart, preventing limit resets during deployments.

## Recording Usage in Real-Time

Every processed request triggers two recording functions:

1. **`recordRequest`** adds the timestamp to the in-memory window and inserts a row into SQLite via `recordUsage` (lines 44-53)
2. **`recordTokens`** performs the same dual-write for token consumption (lines 56-62)

```typescript
import { recordRequest, recordTokens } from './services/ratelimit';

// Log a request for key ID 42 using openrouter/gpt-4o-mini
recordRequest('openrouter', 'gpt-4o-mini', 42);

// Log 350 tokens consumed by that request
recordTokens('openrouter', 'gpt-4o-mini', 42, 350);

```

## Querying Limits and Enforcement

### Dual-Source Counting Strategy

The system implements a resilient counting mechanism that prioritizes database accuracy while maintaining availability:

- **`countPersistedRequests`** and **`sumPersistedTokens`** query the SQLite `rate_limit_usage` table for the specified window (`MINUTE` for RPM/TPM, `DAY` for RPD/TPD), returning `undefined` if the database is unreachable (lines 58-76 and 78-96)
- **`memoryRequestCount`** and **`memoryTokenCount`** prune entries older than the window duration and count the remaining items, serving as fallbacks when persisted counts are unavailable (lines 100-110)

The unified helpers `requestCount` and `tokenCount` automatically prefer persisted values, falling back to in-memory calculations only when necessary (lines 18-23).

### Enforcement Decision Points

The actual limit validation occurs in two specialized functions:

- **`canMakeRequest`** validates against RPM and RPD limits using `requestCount` (lines 38-53)
- **`canUseTokens`** validates against TPM and TPD limits using `tokenCount` (lines 56-74)

```typescript
import { canMakeRequest, canUseTokens } from './services/ratelimit';

const limits = { rpm: 60, rpd: 1000, tpm: 8000, tpd: 100000 };

// Check both request frequency and token budget before forwarding
if (canMakeRequest('openrouter', 'gpt-4o-mini', 42, limits) &&
    canUseTokens('openrouter', 'gpt-4o-mini', 42, estimatedTokens, limits)) {
  // Safe to forward to the LLM provider
}

```

## Provider-Wide Daily Caps

Beyond individual model limits, FreeLLMAPI supports provider-wide daily request quotas that aggregate usage across all models for a specific platform. The system reads the optional environment variable `PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>` (e.g., `PROVIDER_DAILY_REQUEST_CAP_OPENROUTER`), defaulting to 1000 requests per day for OpenRouter if unspecified (lines 87-92).

The functions `getProviderDailyRequestCap`, `providerDailyRequestCount`, and `canUseProvider` enforce these global caps (lines 93-102 and 121-132), preventing scenarios where rotating through multiple models could circumvent daily limits.

## Monitoring Usage with Status Reports

The **`getRateLimitStatus`** function provides real-time visibility into current consumption, returning used counts alongside configured limits for the current minute and day (lines 23-31). This enables dashboard integrations that display remaining quota to users.

```typescript
import { getRateLimitStatus } from './services/ratelimit';

const status = getRateLimitStatus('openrouter', 'gpt-4o-mini', 42, limits);
console.log(`RPM: ${status.rpm.used}/${status.rpm.limit}`);
console.log(`TPD: ${status.tpd.used}/${status.tpd.limit}`);

```

## Summary

- **Composite keys** (`platform:modelId:keyId:type`) isolate rate limit windows per model and API key
- **Dual-layer storage** combines fast in-memory `Map` objects with durable SQLite records in `rate_limit_usage`
- **Automatic fallback** to memory counters occurs when SQLite queries return `undefined`, ensuring continuous enforcement
- **Dedicated functions** `canMakeRequest` and `canUseTokens` handle RPM/RPD and TPM/TPD validation respectively
- **Provider-wide caps** are configurable via `PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>` environment variables

## Frequently Asked Questions

### What happens if the SQLite database becomes unavailable?

When `countPersistedRequests` or `sumPersistedTokens` encounters a database error and returns `undefined`, the system automatically falls back to `memoryRequestCount` and `memoryTokenCount`. These functions prune expired entries from the in-memory window arrays and calculate usage from the remaining data, ensuring rate limiting continues to function even during database outages.

### How does FreeLLMAPI distinguish between request counts and token counts?

The system uses separate storage mechanisms for each metric type. For RPM and RPD, it tracks simple timestamps in the `timestamps` array. For TPM and TPD, it uses the `tokenTimestamps` array containing objects with both timestamp and token count properties. The SQLite persistence layer further distinguishes these via the `kind` column, which stores either 'request' or 'tokens' depending on the usage type being recorded.

### Can I configure provider-wide limits instead of per-model limits?

Yes. Set the environment variable `PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>` where `<PLATFORM>` is the uppercase provider name (e.g., `PROVIDER_DAILY_REQUEST_CAP_OPENROUTER`). If unspecified, OpenRouter defaults to 1000 requests per day. The functions `getProviderDailyRequestCap` and `canUseProvider` enforce these caps across all models for that provider, preventing aggregate quota exhaustion.

### Where are the rate limit windows actually defined in the code?

The core implementation resides in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), which defines the Window interface, composite key generation, and enforcement logic. The database schema for the `rate_limit_usage` table is defined in [`server/src/db/migrations.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/migrations.ts) (lines 100-103), which creates the necessary indexes for efficient time-window queries.