Rate Limit Ledger in FreeLLMAPI: How It Tracks and Enforces API Limits

The Rate Limit Ledger is the internal SQLite-backed mechanism that FreeLLMAPI uses to track, persist, and enforce per-minute and per-day request and token quotas for every API key.

FreeLLMAPI acts as a unified gateway to multiple large language model providers, requiring strict enforcement of usage caps to prevent service interruptions. The Rate Limit Ledger serves as the single source of truth for all rate-limiting decisions, recording every request and token consumption event per (platform – model – API key) tuple while providing fast in-memory fallback capabilities when the database is unavailable.

Core Responsibilities of the Rate Limit Ledger

The ledger manages four primary dimensions of rate limiting, consulting stored data to determine whether new requests may proceed.

Per-Minute and Per-Day Request Tracking

For request-based limits, the ledger maintains separate counters for requests per minute (RPM) and requests per day (RPD). When canMakeRequest() is invoked in server/src/services/ratelimit.ts, it queries the requestCount() function with MINUTE and DAY windows. If the count reaches the configured rpm or rpd thresholds, the function returns false, blocking the request before it reaches the provider.

Token Usage Budgeting

Token limits follow similar logic but sum estimated usage rather than counting discrete requests. The canUseTokens() function checks tokens per minute (TPM) and tokens per day (TPD) via tokenCount(MINUTE) and tokenCount(DAY). This prevents the gateway from exceeding provider-defined token budgets that could trigger account suspension or throttling.

Provider-Wide Daily Caps

Beyond per-key limits, FreeLLMAPI enforces provider-wide caps—such as OpenRouter's free-tier 1,000 requests/day—through the getProviderDailyRequestCap() function. Located in server/src/services/ratelimit.ts (lines 79-99), this guard operates independently of the per-key ledger to prevent the router from saturating shared provider quotas even when individual keys remain under their limits.

Technical Implementation Details

The implementation balances durability with performance through a dual-layer storage strategy.

SQLite Persistence Layer

All rate-limit events are written to the rate_limit_usage table via the recordUsage() function. This SQLite backend ensures that usage counts survive process restarts. A nightly cleanup task removes rows older than 24 hours using DELETE … WHERE created_at_ms <= ?, preventing unbounded table growth while maintaining accurate daily statistics.

In-Memory Fallback

When the database cannot be opened, the ledger falls back to a lightweight in-memory map (windows) that maintains identical API semantics. This withDb() handling ensures that rate limiting continues to function even during database connectivity issues, though without persistence across restarts.

Practical Usage Examples

The following patterns demonstrate how to interact with the Rate Limit Ledger in server/src/services/ratelimit.ts.

Checking Request Permissions

Before forwarding requests to the LLM provider, verify availability against RPM and RPD limits:

import {
  canMakeRequest,
  recordRequest,
} from '@/services/ratelimit.js';

const platform = 'groq';
const modelId = 'llama-70b';
const keyId = 42;
const limits = { rpm: 30, rpd: 1000, tpm: 6_000, tpd: null };

if (canMakeRequest(platform, modelId, keyId, limits)) {
  recordRequest(platform, modelId, keyId);
  // Proceed with API call
} else {
  console.warn('Rate limit exceeded');
}

Managing Token Budgets

For token-intensive operations, validate against TPM and TPD constraints:

import { canUseTokens, recordTokens } from '@/services/ratelimit.js';

const estimatedTokens = 400;

if (canUseTokens(platform, modelId, keyId, estimatedTokens, limits)) {
  // Send request to provider
  recordTokens(platform, modelId, keyId, estimatedTokens);
} else {
  console.warn('Token budget exceeded');
}

Querying Rate Limit Status

Retrieve current usage statistics for monitoring or display:

import { getRateLimitStatus } from '@/services/ratelimit.js';

const status = getRateLimitStatus('groq', 'llama-70b', 42, limits);
console.log(`
  RPM: ${status.rpm.used}/${status.rpm.limit}
  RPD: ${status.rpd.used}/${status.rpd.limit}
  TPM: ${status.tpm.used}/${status.tpm.limit}
`);

Summary

• The Rate Limit Ledger tracks every request and token event per platform-model-key tuple in FreeLLMAPI.
• It enforces RPM, RPD, TPM, and TPD limits through functions in server/src/services/ratelimit.ts.
• SQLite persistence via the rate_limit_usage table ensures counts survive restarts, with nightly cleanup of old records.
• An in-memory fallback (windows map) maintains functionality when the database is unavailable.
• Provider-wide caps complement per-key limits via getProviderDailyRequestCap().

Frequently Asked Questions

What happens when rate limits are exceeded?

When canMakeRequest() or canUseTokens() detects a limit violation in server/src/services/ratelimit.ts, they return false, causing the gateway to reject the request immediately. This prevents the upstream provider from receiving traffic that would violate their quotas, protecting API key health and preventing account suspension.

How does the ledger handle server restarts?

Because the Rate Limit Ledger persists all events to SQLite through recordUsage(), usage counts survive process restarts. The rate_limit_usage table maintains historical data for the last 24 hours, ensuring that daily and per-minute windows remain accurate even after the FreeLLMAPI server reboots.

What is the database schema for the rate limit ledger?

The ledger uses the rate_limit_usage table (defined in server/src/db/migrations/*.sql) to store events with timestamps (created_at_ms). The recordUsage() function inserts rows for each request, while requestCount() and tokenCount() query this table using time-window filters to calculate current usage against configured limits.

How does FreeLLMAPI handle provider-wide caps versus per-key limits?

Per-key limits are enforced by the main ledger functions (canMakeRequest, canUseTokens), while provider-wide caps use a separate getProviderDailyRequestCap() check. This dual-layer approach ensures that even if individual keys have available quota, the router stops sending traffic once the provider's aggregate daily limit (such as OpenRouter's 1,000 request cap) is reached.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}