# How FreeLLMAPI Manages RPM, RPD, TPM, and TPD Rate Limits

> Learn how FreeLLMAPI manages RPM, RPD, TPM, and TPD rate limits using sliding window counters, pre-flight checks, and routing guard-rails to ensure API stability and prevent violations.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-26

---

**FreeLLMAPI enforces per-model and per-provider quotas using a sliding-window counter persisted in SQLite (with an in-memory fallback), applying pre-flight checks and routing guard-rails to prevent limit violations.**

FreeLLMAPI is an open-source routing layer that aggregates multiple LLM providers while enforcing strict **FreeLLMAPI rate limits** to prevent quota exhaustion. The system tracks both request velocity (RPM/RPD) and token consumption (TPM/TPD) across per-model and provider-wide windows, storing usage data in SQLite with graceful degradation to in-memory counters when the database is unavailable.

## Per-Model Rate Limiting (RPM, RPD, TPM, TPD)

Each request or token grant is recorded in the `rate_limit_usage` table, which stores the **platform**, **model ID**, **key ID**, the kind (`'request'` or `'tokens'`), the token count, and a timestamp. The core validation logic lives in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts).

### Request Validation Functions

The entry points for rate limit checks are `canMakeRequest` and `canUseTokens` (lines 12–17 and surrounding implementation). These functions query sliding-window counters before allowing traffic to upstream providers.

```typescript
export function canMakeRequest(
  platform: string,
  modelId: string,
  keyId: number,
  limits: { rpm: number | null; rpd: number | null; tpm: number | null; tpd: number | null },
): boolean {
  const now = Date.now();
  const inFlight = provisionalRequests(platform, modelId, keyId, now);
  if (limits.rpm !== null && requestCount(platform, modelId, keyId, MINUTE, now) + inFlight >= limits.rpm) return false;
  if (limits.rpd !== null && requestCount(platform, modelId, keyId, DAY, now) + inFlight >= limits.rpd) return false;
  return true;
}

```

```typescript
export function canUseTokens(
  platform: string,
  modelId: string,
  keyId: number,
  estimatedTokens: number,
  limits: { tpm: number | null; tpd: number | null },
): boolean {
  const now = Date.now();
  const inFlight = provisionalTokens(platform, modelId, keyId, now);
  if (limits.tpm !== null) {
    const used = tokenCount(platform, modelId, keyId, MINUTE, now);
    if (used + inFlight + estimatedTokens > limits.tpm) return false;
  }
  if (limits.tpd !== null) {
    const used = tokenCount(platform, modelId, keyId, DAY, now);
    if (used + inFlight + estimatedTokens > limits.tpd) return false;
  }
  return true;
}

```

### Sliding-Window Implementation

The window definitions use `MINUTE = 60_000`ms for per-minute limits and `msSinceUtcMidnight` for per-day limits to align with UTC boundaries. The `requestCount` and `tokenCount` helpers first attempt to read from persisted SQLite counters (`countPersistedRequests` / `sumPersistedTokens`) and transparently fall back to in-memory sliding windows (`memoryRequestCount` / `memoryTokenCount`) if the database is unavailable.

## Provider-Wide Quota Caps

Some providers (such as OpenRouter’s free tier) expose a single quota shared across all models. FreeLLMAPI enforces these global caps separately from per-model limits.

### Environment-Based Configuration

Default caps are defined in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) (lines 61–78) and can be overridden via environment variables:
- `PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>`
- `PROVIDER_MINUTE_REQUEST_CAP_<PLATFORM>`
- `PROVIDER_DAILY_TOKEN_CAP_<PLATFORM>`

### Aggregate Enforcement Functions

The functions `getProviderDailyRequestCap`, `getProviderMinuteRequestCap`, and `getProviderDailyTokenCap` return the configured limits. These feed into `canUseProvider`, `canUseProviderMinute`, and `canUseProviderTokens`, which aggregate usage across **all** models belonging to the same account/key rather than checking a single model in isolation.

## Concurrency Guardrails

FreeLLMAPI also limits the number of *in-flight* requests per key, which protects providers that meter concurrency rather than per-minute throughput. The limit is read from `MAX_CONCURRENT_REQUESTS_PER_KEY_<PLATFORM>` (with a generic fallback) in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) (lines 98–106) and enforced via `canUseKeyConcurrency`.

## Routing Guardrails and Fractional Utilization

To avoid abrupt cutoffs, the scoring engine uses fractional utilization to gently demote keys as they approach limits. This is implemented in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts) (lines 73–84) via `buildWindowUsageSnapshot`, which performs a single grouped query over the last minute and day for all keys, then caches the result for a 5-second TTL.

The `modelWindowUsedFraction` function calculates pressure by comparing current usage against limits. This value feeds into `rateWindowHeadroomFactor` in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts) (lines 94–104), which applies a smooth penalty to a model’s score as its quota consumes, preserving throughput for healthier alternatives rather than blocking traffic only after the limit is hit.

## Cooldown Escalation After HTTP 429

When a provider returns a 429 status code, FreeLLMAPI places the model+key combination on a cooldown. The logic in `getCooldownDecisionForLimit` ([`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), lines 66–78) consults a ladder (`COOLDOWN_DURATIONS`) that escalates up to 24 hours when the daily quota appears exhausted, or falls back to a 90-second bench for transient per-minute failures. This prevents hammering providers that have already signaled exhaustion.

## Summary

- **Dual-layer persistence**: Rate limit counters are stored in SQLite with an automatic in-memory fallback for high availability.
- **Per-model and provider-wide enforcement**: Separate tracks validate limits for individual models and aggregate quotas across provider accounts.
- **Sliding-window logic**: RPM/TPM use 60-second windows; RPD/TPD align to UTC midnight boundaries.
- **Pre-flight validation**: `canMakeRequest` and `canUseTokens` reject requests before they reach upstream providers, conserving quota.
- **Graceful degradation**: Routing guardrails use fractional utilization (`modelWindowUsedFraction`) and headroom factors to smoothly steer traffic away from saturated keys.
- **Exponential backoff**: 429 responses trigger escalating cooldowns up to 24 hours to handle daily limit exhaustion.

## Frequently Asked Questions

### How does FreeLLMAPI handle database failures during rate limit checks?

When the SQLite database is unavailable, the `requestCount` and `tokenCount` functions automatically fall back to in-memory sliding-window counters (`memoryRequestCount` and `memoryTokenCount`). This ensures that rate limiting continues to function even during database outages, although counts will reset if the server process restarts.

### What is the difference between per-model and provider-wide rate limits in FreeLLMAPI?

Per-model limits (enforced by `canMakeRequest` and `canUseTokens`) track usage for a specific model ID and key combination, while provider-wide caps (enforced by `canUseProvider` and related functions) aggregate usage across all models belonging to the same provider account. Provider limits are configured via environment variables like `PROVIDER_DAILY_REQUEST_CAP_<PLATFORM>` and protect shared quotas such as OpenRouter’s free tier.

### How does the routing guardrail prevent quota exhaustion before limits are hit?

The router calls `modelWindowUsedFraction` to calculate a utilization percentage between 0 and 1. This feeds into `rateWindowHeadroomFactor` in [`server/src/services/scoring.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/scoring.ts), which applies a diminishing multiplier to the model’s selection score as utilization increases. By the time a key reaches its RPM or TPM limit, its score has degraded sufficiently that the router prefers alternative keys, preventing hard rejections.

### What happens when a provider returns a 429 error?

The system invokes `getCooldownDecisionForLimit` to determine a backoff duration. Based on the `COOLDOWN_DURATIONS` ladder, the model+key is benched for 90 seconds if the error appears transient, or up to 24 hours if the daily quota is exhausted. Subsequent routing decisions consult this cooldown state via `setCooldown` and reject traffic to that key until the period expires.