How to Debug Routing Failures or Rate Limit Issues in FreeLLMAPI

To debug routing failures and rate limit issues in FreeLLMAPI, inspect the active fallback chain with getOrderedFusionChain(), check rate limit status via getRateLimitStatus(), and catch RouteError to read the diagnostics array that explains exactly why each model was skipped.

FreeLLMAPI routes every request through a bandit-style fallback chain that scores models, selects usable API keys, and enforces per-model and per-key limits. When you encounter "All models exhausted" (429) or rapid 429/402/403 responses, the failure typically originates from one of three architectural layers. Understanding how to query the internal state of the routing system allows you to pinpoint whether the issue is a missing vision model, an exhausted quota, or a stale cooldown.

Understanding the Routing Architecture

FreeLLMAPI's routing pipeline consists of three distinct layers that can each trigger failures. Knowing which layer is rejecting your request is the first step toward resolution.

The Routing Chain Layer

The router.ts file orders models by the selected strategy (priority, balanced, smartest, etc.) and then attempts to find a valid key. If no model satisfies the request constraints—for example, a vision request when no vision-enabled model is available—the chain aborts before touching any keys.

Key Selection and Cooldown

The selectKeyForModel function in router.ts rotates across enabled keys for a selected model, checking provider-level caps, per-key cooldowns, and token limits. When all keys for a model are on cooldown or over quota, the router silently skips that model and moves to the next in the chain.

Rate Limit Tracking

The ratelimit.ts module persists request and token counts, applies sliding-window limits, records cooldowns, and learns provider limits from error messages. When a 429 (RPM/RPD) or 402/403 (payment-required/forbidden) occurs, the system imposes a cooldown that blocks the model or key.

Debugging Method 1: Inspect the Active Fallback Chain

When requests fail immediately, verify that the routing chain actually contains valid models. The getOrderedFusionChain() function in router.ts (lines 280-285) returns the models that could serve the request right now, ordered by the current routing strategy.

import { getOrderedFusionChain } from './router';

// Returns models that *could* serve the request right now
const chain = getOrderedFusionChain();
console.log(chain);

If this array is empty or lacks vision/tool support, the router aborts before attempting any keys.

Debugging Method 2: Check Rate Limit Status and Penalties

FreeLLMAPI demotes models after 429 errors using a penalty counter. You can inspect current penalties and exact usage counters to identify blocked resources.

View Per-Model Penalties

The recordRateLimitHit function (lines 95-107) in router.ts increments penalties that push models toward the end of the chain. Access these via:

import { getAllPenalties } from './router';

console.log(getAllPenalties());

High penalty values make models "invisible" to subsequent requests.

Query Key Usage Counters

To see exact rate limit consumption for a specific key, use getRateLimitStatus from ratelimit.ts (lines 80-93):

import { getRateLimitStatus } from './ratelimit';

const status = getRateLimitStatus(
  'openai',            // platform
  'gpt-4o-mini',      // model_id
  42,                  // key_id (from the DB)
  {
    rpm: 60,
    rpd: 5000,
    tpm: null,
    tpd: null,
  }
);
console.log(status);

This returns used counters for minute and day windows, revealing whether a key has hit its RPM or RPD limits.

Debugging Method 3: Verify Cooldown States

When providers return 429 errors, the router calls setCooldown (lines 50-55) in ratelimit.ts. Check if a specific model/key pair is still blocked:

import { isOnCooldown } from './ratelimit';

const onCooldown = isOnCooldown('openai', 'gpt-4o-mini', 42);
console.log(`Cooldown? ${onCooldown}`);

If this returns true, the router will skip that key until the expiry time passes.

Debugging Method 4: Capture Routing Diagnostics

The selectKeyForModel function collects diagnostic strings whenever it rejects a key (e.g., cooldown, rpm/rpd-limit, tpm/tpd-limit). When the chain is exhausted, the thrown RouteError contains these diagnostics. The RouteError class is defined in router.ts (lines 16-28).

import { routeRequest, RouteError } from './router';

try {
  const result = routeRequest(1500); // estimated token count
} catch (e) {
  if (e instanceof RouteError) {
    console.error('Routing failed:', e.message);
    console.error('Diagnostics:', e.diagnostics?.join('\n'));
  } else {
    throw e;
  }
}

The diagnostics array reveals exactly which checks failed for each model in the chain.

Debugging Method 5: Learn Limits from Provider Errors

If providers return "Limit N" errors (e.g., Groq 413), FreeLLMAPI parses and persists these via learnLimitFromError (lines 48-58) in ratelimit.ts. Force a re-learn to update stale limits:

import { learnLimitFromError } from './ratelimit';

// Simulate an error response with a limit message
const fakeError = { message: 'Error: limit 3000 tokens per minute' };
learnLimitFromError(123, fakeError); // 123 = model_db_id

Common "All Models Exhausted" Checklist

When you encounter the generic 429 message, verify these six conditions in sequence:

1. Active chain – getOrderedFusionChain() returns at least one model.
2. Vision/Tools support – For image requests, confirm hasEnabledVisionModel() returns true (see router.ts lines 94-99).
3. Key health – At least one api_keys row has status = 'healthy'.
4. Cooldowns – isOnCooldown returns false for each candidate key.
5. RPM/RPD – getRateLimitStatus(...).rpm.used < rpm and rpd.used < rpd.
6. TPM/TPD – getRateLimitStatus(...).tpm.used + estimatedTokens <= tpm (if tpm is set).

Failures in any of these checks appear in the RouteError.diagnostics array.

Practical Debugging Examples

Print Full Routing Diagnostics

Run this to surface exactly why a request fails:

import { routeRequest, RouteError } from './router';

async function debugRequest() {
  try {
    const res = routeRequest(2000, undefined, undefined, true, false);
    console.log('✅ Route succeeded:', res);
  } catch (e) {
    if (e instanceof RouteError) {
      console.error('❌ Routing failed (status', e.status, ')');
      console.error('Diagnostics:\n', e.diagnostics?.join('\n'));
    } else {
      console.error('Unexpected error:', e);
    }
  }
}
debugRequest();

Manually Reset a Cooldown

Use this after sandbox testing to clear a stuck cooldown:

import { setCooldown } from './ratelimit';

// Remove the 2-minute cooldown for key 42 on model gpt-4o-mini
setCooldown('openai', 'gpt-4o-mini', 42, 0);
console.log('Cooldown cleared');

Force Limit Re-Learning

Update the stored limit after receiving a new provider error:

import { learnLimitFromError } from './ratelimit';

const err = { message: 'Limit 5000 requests per day' };
learnLimitFromError(42, err); // 42 = model_db_id

Summary

• FreeLLMAPI uses a three-layer architecture: routing chain ordering, key selection with cooldown checks, and persistent rate limit tracking.
• Inspect the chain with getOrderedFusionChain() to verify models are available before troubleshooting keys.
• Check penalties via getAllPenalties() and rate usage via getRateLimitStatus() to identify blocked resources.
• Verify cooldowns using isOnCooldown() to confirm if keys are temporarily disabled.
• Read diagnostics by catching RouteError and inspecting the diagnostics array, which lists exact failure reasons for each model.
• Learn new limits with learnLimitFromError() when providers change their quotas or return limit errors.

Frequently Asked Questions

What does "All models exhausted" mean in FreeLLMAPI?

This error indicates that the routing chain iterated through every available model but failed to find a usable API key for any of them. According to the source code in router.ts, this occurs when selectKeyForModel rejects all keys for every model due to cooldowns, rate limits, or missing capabilities. The RouteError thrown contains a diagnostics array that specifies exactly why each model was skipped.

How do I check if a specific API key is on cooldown?

Import isOnCooldown from ratelimit.ts and pass the platform string, model ID, and key ID. The function queries the SQLite-backed cooldown store (defined in the database layer) and returns a boolean indicating whether the key is currently blocked. This check happens automatically during routing, but you can call it manually to verify key health before submitting requests.

Where does FreeLLMAPI store rate limit usage data?

Rate limit counters persist in SQLite tables defined in server/src/db/model-pricing.ts, specifically rate_limit_usage and rate_limit_cooldowns. The ratelimit.ts service manages sliding-window counters for RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day), updating these records after each request and checking them during the selectKeyForModel phase.

How can I reset rate limit penalties for testing?

To clear penalties during development, import setCooldown from ratelimit.ts and set the duration to 0 for the specific platform, model, and key ID. Alternatively, you can manually adjust the penalty counters in memory if you have access to the internal state, though the cooldown reset is the safer production-adjacent method. For penalty scores specifically, restart the server process to clear the in-memory penalty map maintained by recordRateLimitHit.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}