How to Troubleshoot 429 Rate Limit Errors in FreeLLMAPI: A Complete Guide

FreeLLMAPI automatically handles 429 errors by backing off, marking keys as rate-limited, and retrying fallback models, but you can diagnose issues by inspecting response headers, querying the SQLite usage database, and checking penalty scores in the router.

The FreeLLMAPI repository (tashfeenahmed/freellmapi) implements an intelligent routing system that manages provider rate limits through a combination of sliding-window counters, cooldown mechanisms, and dynamic penalty scoring. When you encounter 429 Too Many Requests errors, understanding the internal flow—from the router's selection logic to the rate-limit ledger's SQLite backing—enables you to pinpoint whether the bottleneck is a per-minute request cap, a daily quota, or a temporary provider cooldown.

Understanding the Rate Limit Architecture

FreeLLMAPI routes every request through a centralized router that evaluates provider health and rate-limit state before dispatching. The system relies on three core components working in concert: the router logic, the rate-limit ledger, and provider-wide caps.

The Router Component (router.ts)

In server/src/services/router.ts, the router selects the best-available model using four critical checks: canMakeRequest, canUseTokens, isOnCooldown, and a dynamic penalty system. When a provider returns a 429, the router immediately calls recordRateLimitHit(modelDbId) to demote that model in the fallback chain and invokes setCooldown(keyId) to prevent immediate reuse.

The Rate Limit Ledger (ratelimit.ts)

The server/src/services/ratelimit.ts file maintains in-memory sliding-window counters backed by SQLite. It records every request and token usage in the rate_limit_usage table, prunes expired entries, and answers capacity queries via canMakeRequest and canUseTokens. The isOnCooldown function checks whether a key remains temporarily blacklisted after a recent 429.

Provider-Wide Daily Caps

Some providers impose shared daily quotas across all models. The getProviderDailyRequestCap function in ratelimit.ts reads environment variables (e.g., PROVIDER_DAILY_REQUEST_CAP_OPENROUTER) or defaults defined in DEFAULT_PROVIDER_DAILY_REQUEST_CAPS, enforcing these limits regardless of individual key health.

How the 429 Fallback Flow Works

When your request triggers a rate limit, FreeLLMAPI executes the following sequence before returning a response:

1. Selection: The router picks a key and validates it via canMakeRequest and canUseTokens.
2. Error Capture: The provider adapter catches the 429 and propagates it to the router.
3. Penalty Application: The router calls recordRateLimitHit, adding a penalty that decays over time (DECAY_INTERVAL_MS = 2 minutes).
4. Cooldown: The key enters a cooldown state via setCooldown, preventing immediate reuse.
5. Retry: The router attempts the next model in the fallback chain (up to 20 attempts).
6. Final Response: If all models exhaust, the original 429 reaches the client, often with X-Fallback-Attempts headers indicating retry count.

Diagnostic Methods for 429 Errors

To troubleshoot effectively, inspect the following data sources in order of granularity:

Inspect Response Headers

Every response includes X-Routed-Via (provider/model) and X-Fallback-Attempts (retry count). Use these headers to determine if a request succeeded after retries or failed entirely:

curl -i -H "Authorization: Bearer <your-key>" \
     http://localhost:3001/v1/chat/completions

Look for X-Routed-Via: openrouter/gpt-4 and X-Fallback-Attempts: 3 to identify which provider ultimately served the request.

Query the SQLite Usage Database

The rate_limit_usage table tracks every request and token count. Query recent usage to identify which keys approach their limits:

SELECT platform, model_id, key_id, 
       SUM(CASE WHEN kind='request' THEN 1 ELSE 0 END) AS reqs,
       SUM(CASE WHEN kind='tokens' THEN tokens ELSE 0 END) AS toks
FROM rate_limit_usage
WHERE created_at_ms > strftime('%s','now')*1000 - 86400000
GROUP BY platform, model_id, key_id;

Run this via the bundled SQLite CLI or any database viewer while the server is running.

Check the In-Memory Windows

While debugging, inspect the windows Map in ratelimit.ts to see current sliding-window counters. Access this via Node.js inspection only—never expose this data in production:

// Debugging only: attach to running process
console.log(global.windows);

Review Penalty Scores

Call getAllPenalties() from server/src/services/router.ts to see models with recent 429 hits. High penalty scores push models down the fallback order. You can invoke this from a temporary debug endpoint or REPL:

import { getAllPenalties } from './server/src/services/router.ts';
console.table(getAllPenalties());

Verify Environment Variables

Check for provider-specific caps that might override defaults:

echo $PROVIDER_DAILY_REQUEST_CAP_OPENROUTER

If unset, FreeLLMAPI falls back to internal defaults defined in model-pricing.ts.

Common Causes and Solutions

Cause	Diagnostic Signal	Fix
Per-minute request limit (RPM) exceeded	canMakeRequest returns false; high reqs count in SQL query	Reduce concurrency, add X-Session-Id for sticky sessions, or increase provider quota
Per-day request quota (RPD) exhausted	Provider caps hit in getProviderDailyRequestCap	Add more keys for that provider, reorder fallback chain, or upgrade provider tier
Token budget (TPM/TPD) exceeded	canUseTokens returns false	Lower max_tokens parameter or spread usage across keys with larger budgets
Penalty buildup	High scores in getAllPenalties()	Wait for decay (2 minutes) or manually clear penalties
Stale cooldown flag	Key remains unavailable after provider recovery	Restart server to clear in-memory cooldowns or wait for COOLDOWN_MS timeout
Provider-wide daily cap	OpenRouter or similar shared quota exhausted	Add another provider or set PROVIDER_DAILY_REQUEST_CAP_OPENROUTER=0 to disable (surfaces 429 earlier)

Practical Debugging Examples

Debug Script for Usage Analysis

Run this Node.js script from the repo root to print per-key usage statistics:

import { getDb } from './server/src/db/index.js';

const db = getDb();
const dayAgo = Date.now() - 24 * 60 * 60 * 1000;

const rows = db.prepare(`
  SELECT platform, model_id, key_id,
         SUM(CASE WHEN kind='request' THEN 1 ELSE 0 END) AS reqs,
         SUM(CASE WHEN kind='tokens'  THEN tokens ELSE 0 END) AS toks
    FROM rate_limit_usage
   WHERE created_at_ms > ?
   GROUP BY platform, model_id, key_id
`).all(dayAgo);

console.table(rows);

Detecting 429 and Logging Penalties

Add this to your error handling to track rate limits in real-time:

import { recordRateLimitHit, getAllPenalties } from './server/src/services/router.ts';

// Inside catch block
if (error.status === 429) {
  const modelId = error.modelId;
  recordRateLimitHit(modelId);
  console.warn(`429 from model ${modelId}; penalty applied`);
}

// Log penalties every 5 minutes
setInterval(() => {
  console.table(getAllPenalties());
}, 5 * 60 * 1000);

Manually Clearing a Cooldown (Development Only)

For local debugging, you can force-clear a cooldown:

import { clearCooldown } from './server/src/services/ratelimit.ts';
clearCooldown(3); // Removes cooldown for key ID 3

Warning: Never expose clearCooldown in production endpoints.

Summary

• FreeLLMAPI handles 429 errors automatically through router.ts, which applies penalties and cooldowns before retrying fallback models.
• Inspect response headers (X-Routed-Via, X-Fallback-Attempts) to trace which provider served your request.
• Query the rate_limit_usage table in SQLite to verify per-key request and token counts against limits defined in model-pricing.ts.
• Check getAllPenalties() in server/src/services/router.ts to identify models demoted due to repeated 429s.
• Adjust environment variables like PROVIDER_DAILY_REQUEST_CAP_OPENROUTER to customize provider-wide daily quotas.
• Use sticky sessions (X-Session-Id) to reduce rate limit contention when running high-concurrency workloads.

Frequently Asked Questions

What does the X-Routed-Via header indicate?

The X-Routed-Via header reveals the specific provider and model that ultimately processed your request, such as openrouter/gpt-4 or anthropic/claude-3. When X-Fallback-Attempts is greater than zero, this header shows the final successful provider after retries, helping you identify which keys in the chain returned 429 errors.

How long does a rate limit cooldown last?

By default, keys remain on cooldown for 30 seconds after a 429 error, as defined by the COOLDOWN_MS constant in server/src/services/ratelimit.ts. Penalties decay every 2 minutes (DECAY_INTERVAL_MS), gradually restoring the model's priority in the fallback chain.

Can I disable provider daily caps?

Yes, set the environment variable PROVIDER_DAILY_REQUEST_CAP_<PROVIDER> to 0 to disable the cap for that specific provider. For example, PROVIDER_DAILY_REQUEST_CAP_OPENROUTER=0 removes the daily request limit check, though this will cause FreeLLMAPI to surface 429 errors immediately from the provider rather than internally throttling.

Why am I still getting 429 errors after adding more keys?

If you have added keys but still receive 429s, check for provider-wide daily caps that apply across all models, verify that new keys are marked as healthy in the dashboard (client/src/pages/Keys.tsx), and ensure no penalty buildup is demoting your new keys via getAllPenalties(). Additionally, confirm that the keys support the specific models in your fallback chain by checking model-pricing.ts for compatible rpm_limit and rpd_limit configurations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}