# How FreeLLMAPI Manages Rate Limit Counter Persistence: A Hybrid SQLite Approach

> Discover how FreeLLMAPI ensures rate limit counter persistence through a hybrid SQLite and in-memory approach. Learn how it survives restarts without sacrificing speed.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-06-27

---

**FreeLLMAPI implements a dual-layer persistence strategy that writes every request and token usage record to a SQLite database while maintaining hot in-memory sliding windows, ensuring rate limit counters survive process restarts without degrading real-time request throughput.**

FreeLLMAPI is an open-source LLM routing proxy that enforces provider rate limits across multiple AI platforms. According to the source code in the `tashfeenahmed/freellmapi` repository, the system manages **rate limit counter persistence** through a hybrid architecture that combines durable SQLite storage with high-performance in-memory tracking.

## Database Schema for Rate Limit Storage

The persistence layer relies on two tables created by the migration file [`server/src/db/migrations/20260101_000000_legacy_baseline.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/migrations/20260101_000000_legacy_baseline.ts).

### The rate_limit_usage Table

This table stores granular records for every request and token consumption event. The schema (lines 104-112) includes fields for platform, model, key identification, usage kind, and precise timestamps:

```sql
CREATE TABLE IF NOT EXISTS rate_limit_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    platform TEXT NOT NULL,
    model_id TEXT NOT NULL,
    key_id INTEGER NOT NULL,
    kind TEXT NOT NULL CHECK (kind IN ('request','tokens')),
    tokens INTEGER NOT NULL DEFAULT 0,
    created_at_ms INTEGER NOT NULL,
    created_at TEXT NOT NULL DEFAULT (datetime('now'))
);

```

### The rate_limit_cooldowns Table

The same migration creates `rate_limit_cooldowns` to store per-key expiration timestamps when models enter a cooldown period after hitting rate limits.

## Recording Usage to the Database

In [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), the `recordRequest` function (lines 44-55) persists usage data immediately after updating in-memory windows. This ensures the SQLite database remains the authoritative source of truth:

```typescript
export function recordRequest(platform, modelId, keyId) {
    const now = Date.now();
    // …push timestamps into in‑memory windows…
    recordUsage(platform, modelId, keyId, 'request', 0, now);
    clearNullLimitHits(platform, modelId, keyId);
}

```

The `recordTokens` function follows an identical pattern, writing token counts to the same table with `kind` set to `'tokens'`.

## Querying Persisted Counters

When performing limit checks, the system queries the database to calculate usage within sliding windows. The `countPersistedRequests` function (lines 58-76) demonstrates this aggregation:

```typescript
function countPersistedRequests(..., windowMs, now) {
    return withDb(db => {
        const row = db.prepare(`
            SELECT COUNT(*) AS used
            FROM rate_limit_usage
            WHERE platform = ?
              AND model_id = ?
              AND key_id = ?
              AND kind = 'request'
              AND created_at_ms > ?
        `).get(platform, modelId, keyId, now - windowMs);
        return row.used;
    });
}

```

Token consumption uses `sumPersistedTokens`, which applies `SUM(tokens)` instead of `COUNT(*)`. Both functions utilize the `withDb` helper to ensure connection management.

## In-Memory Fallback and Sliding Windows

If the database is unavailable (when `withDb` returns `undefined`), FreeLLMAPI falls back to pure in-memory tracking. The module maintains a `Map<string, Window>` keyed by `"platform:modelId:keyId:type"` where type can be `rpm`, `rpd`, `tpm`, or `tpd` (lines 100-115).

The `memoryRequestCount` and `memoryTokenCount` functions provide the same sliding-window logic without SQLite overhead, ensuring rate limiting continues even during database outages.

### Provider-Wide Daily Caps

Some providers (such as OpenRouter's free tier) impose shared daily request caps across all models for a single API key. FreeLLMAPI handles this through `providerDailyRequestCount` (lines 221-234), which aggregates persisted rows across all models for a given key. When the database is unreachable, it falls back to summing the per-model in-memory windows.

## Persisting Cooldowns After Errors

After receiving 429, 402, or 403 responses, the router may bench a specific model and key combination. The expiry timestamp is stored in `rate_limit_cooldowns` via `persistCooldown`. On each request, `isOnCooldown` (lines 311-340) first queries the database and updates the in-memory `cooldowns` map if a persisted record exists, ensuring cooldowns survive restarts.

## Practical Implementation Example

The following example demonstrates how to check limits before making requests and persist usage afterward:

```typescript
import { canMakeRequest, canUseTokens, recordRequest, recordTokens } from './services/ratelimit.js';

const limits = {
  rpm: 20,   // requests per minute (may be null)
  rpd: 200,  // requests per day
  tpm: 250_000,
  tpd: null, // tokens per day unknown
};

if (canMakeRequest('openrouter', 'deepseek/deepseek-v3.1:free', 1, limits)) {
  // proceed with the call …
  // after a successful call:
  recordRequest('openrouter', 'deepseek/deepseek-v3.1:free', 1);
} else {
  console.warn('Rate limit exceeded – skip this model/key');
}

// Token‑budget check before sending a large prompt
const estimatedTokens = 1500;
if (canUseTokens('groq', 'llama-3.3-70b-versatile', 2, estimatedTokens, {
  tpm: 8000,
  tpd: null,
})) {
  // send the request
  recordTokens('groq', 'llama-3.3-70b-versatile', 2, estimatedTokens);
} else {
  console.warn('Token budget exceeded');
}

```

## Summary

- **Hybrid architecture**: FreeLLMAPI combines SQLite persistence with in-memory sliding windows for both durability and performance.
- **Granular storage**: Every request and token event writes to `rate_limit_usage`, while cooldown states live in `rate_limit_cooldowns`.
- **Automatic fallback**: When `withDb` returns undefined, the system seamlessly switches to `memoryRequestCount` and `memoryTokenCount` without dropping limit enforcement.
- **Provider-aware**: Special handling for provider-wide daily caps aggregates usage across models when querying the database.
- **Source location**: All core logic resides in [`server/src/services/ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts), with schema definitions in [`server/src/db/migrations/20260101_000000_legacy_baseline.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/migrations/20260101_000000_legacy_baseline.ts).

## Frequently Asked Questions

### What database does FreeLLMAPI use for rate limit persistence?

FreeLLMAPI uses **SQLite** for all rate limit persistence. The database stores individual usage records in the `rate_limit_usage` table and cooldown timestamps in `rate_limit_cooldowns`, accessed through the `withDb` helper defined in the database module.

### How does FreeLLMAPI handle rate limits when the database is offline?

The system implements a **graceful degradation strategy**. If the `withDb` helper returns `undefined` (indicating the SQLite connection is unavailable), FreeLLMAPI falls back to `memoryRequestCount` and `memoryTokenCount` functions that maintain sliding windows in a JavaScript `Map`. This ensures rate limiting continues to function even during database outages.

### Does FreeLLMAPI support provider-wide daily request limits?

Yes. For providers like OpenRouter that enforce daily caps across all models for a given API key, FreeLLMAPI uses `providerDailyRequestCount` (lines 221-234). This function aggregates persisted records across all models when the database is available, or sums in-memory windows as a fallback.

### How are cooldowns persisted after hitting a rate limit?

When the router encounters a 429, 402, or 403 response, it calls `persistCooldown` to write the expiration timestamp to the `rate_limit_cooldowns` table. On subsequent requests, `isOnCooldown` first checks this database table and synchronizes the result to an in-memory cache, ensuring cooldowns survive process restarts while remaining fast to query.