# How to Debug Routing Failures or Rate Limit Issues in FreeLLMAPI

> Debug routing failures and rate limit issues in FreeLLMAPI. Use getOrderedFusionChain, getRateLimitStatus, and catch RouteError to diagnose model skips with diagnostic details.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-29

---

**To debug routing failures and rate limit issues in FreeLLMAPI, inspect the active fallback chain with `getOrderedFusionChain()`, check rate limit status via `getRateLimitStatus()`, and catch `RouteError` to read the `diagnostics` array that explains exactly why each model was skipped.**

FreeLLMAPI routes every request through a bandit-style fallback chain that scores models, selects usable API keys, and enforces per-model and per-key limits. When you encounter "All models exhausted" (429) or rapid 429/402/403 responses, the failure typically originates from one of three architectural layers. Understanding how to query the internal state of the routing system allows you to pinpoint whether the issue is a missing vision model, an exhausted quota, or a stale cooldown.

## Understanding the Routing Architecture

FreeLLMAPI's routing pipeline consists of three distinct layers that can each trigger failures. Knowing which layer is rejecting your request is the first step toward resolution.

### The Routing Chain Layer

The [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) file orders models by the selected strategy (`priority`, `balanced`, `smartest`, etc.) and then attempts to find a valid key. If no model satisfies the request constraints—for example, a vision request when no vision-enabled model is available—the chain aborts before touching any keys.

### Key Selection and Cooldown

The `selectKeyForModel` function in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) rotates across enabled keys for a selected model, checking provider-level caps, per-key cooldowns, and token limits. When all keys for a model are on cooldown or over quota, the router silently skips that model and moves to the next in the chain.

### Rate Limit Tracking

The [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) module persists request and token counts, applies sliding-window limits, records cooldowns, and learns provider limits from error messages. When a 429 (RPM/RPD) or 402/403 (payment-required/forbidden) occurs, the system imposes a cooldown that blocks the model or key.

## Debugging Method 1: Inspect the Active Fallback Chain

When requests fail immediately, verify that the routing chain actually contains valid models. The `getOrderedFusionChain()` function in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) (lines [280-285](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts#L280-L285)) returns the models that could serve the request right now, ordered by the current routing strategy.

```typescript
import { getOrderedFusionChain } from './router';

// Returns models that *could* serve the request right now
const chain = getOrderedFusionChain();
console.log(chain);

```

If this array is empty or lacks vision/tool support, the router aborts before attempting any keys.

## Debugging Method 2: Check Rate Limit Status and Penalties

FreeLLMAPI demotes models after 429 errors using a penalty counter. You can inspect current penalties and exact usage counters to identify blocked resources.

### View Per-Model Penalties

The `recordRateLimitHit` function (lines [95-107](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts#L95-L107)) in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) increments penalties that push models toward the end of the chain. Access these via:

```typescript
import { getAllPenalties } from './router';

console.log(getAllPenalties());

```

High penalty values make models "invisible" to subsequent requests.

### Query Key Usage Counters

To see exact rate limit consumption for a specific key, use `getRateLimitStatus` from [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) (lines [80-93](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts#L80-L93)):

```typescript
import { getRateLimitStatus } from './ratelimit';

const status = getRateLimitStatus(
  'openai',            // platform
  'gpt-4o-mini',      // model_id
  42,                  // key_id (from the DB)
  {
    rpm: 60,
    rpd: 5000,
    tpm: null,
    tpd: null,
  }
);
console.log(status);

```

This returns used counters for minute and day windows, revealing whether a key has hit its RPM or RPD limits.

## Debugging Method 3: Verify Cooldown States

When providers return 429 errors, the router calls `setCooldown` (lines [50-55](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts#L50-L55)) in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts). Check if a specific model/key pair is still blocked:

```typescript
import { isOnCooldown } from './ratelimit';

const onCooldown = isOnCooldown('openai', 'gpt-4o-mini', 42);
console.log(`Cooldown? ${onCooldown}`);

```

If this returns `true`, the router will skip that key until the expiry time passes.

## Debugging Method 4: Capture Routing Diagnostics

The `selectKeyForModel` function collects diagnostic strings whenever it rejects a key (e.g., `cooldown`, `rpm/rpd-limit`, `tpm/tpd-limit`). When the chain is exhausted, the thrown `RouteError` contains these diagnostics. The `RouteError` class is defined in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) (lines [16-28](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts#L16-L28)).

```typescript
import { routeRequest, RouteError } from './router';

try {
  const result = routeRequest(1500); // estimated token count
} catch (e) {
  if (e instanceof RouteError) {
    console.error('Routing failed:', e.message);
    console.error('Diagnostics:', e.diagnostics?.join('\n'));
  } else {
    throw e;
  }
}

```

The `diagnostics` array reveals exactly which checks failed for each model in the chain.

## Debugging Method 5: Learn Limits from Provider Errors

If providers return "Limit N" errors (e.g., Groq 413), FreeLLMAPI parses and persists these via `learnLimitFromError` (lines [48-58](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/ratelimit.ts#L48-58)) in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts). Force a re-learn to update stale limits:

```typescript
import { learnLimitFromError } from './ratelimit';

// Simulate an error response with a limit message
const fakeError = { message: 'Error: limit 3000 tokens per minute' };
learnLimitFromError(123, fakeError); // 123 = model_db_id

```

## Common "All Models Exhausted" Checklist

When you encounter the generic 429 message, verify these six conditions in sequence:

1. **Active chain** – `getOrderedFusionChain()` returns at least one model.
2. **Vision/Tools support** – For image requests, confirm `hasEnabledVisionModel()` returns `true` (see [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts) lines [94-99](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts#L94-L99)).
3. **Key health** – At least one `api_keys` row has `status = 'healthy'`.
4. **Cooldowns** – `isOnCooldown` returns `false` for each candidate key.
5. **RPM/RPD** – `getRateLimitStatus(...).rpm.used < rpm` and `rpd.used < rpd`.
6. **TPM/TPD** – `getRateLimitStatus(...).tpm.used + estimatedTokens <= tpm` (if `tpm` is set).

Failures in any of these checks appear in the `RouteError.diagnostics` array.

## Practical Debugging Examples

### Print Full Routing Diagnostics

Run this to surface exactly why a request fails:

```typescript
import { routeRequest, RouteError } from './router';

async function debugRequest() {
  try {
    const res = routeRequest(2000, undefined, undefined, true, false);
    console.log('✅ Route succeeded:', res);
  } catch (e) {
    if (e instanceof RouteError) {
      console.error('❌ Routing failed (status', e.status, ')');
      console.error('Diagnostics:\n', e.diagnostics?.join('\n'));
    } else {
      console.error('Unexpected error:', e);
    }
  }
}
debugRequest();

```

### Manually Reset a Cooldown

Use this after sandbox testing to clear a stuck cooldown:

```typescript
import { setCooldown } from './ratelimit';

// Remove the 2-minute cooldown for key 42 on model gpt-4o-mini
setCooldown('openai', 'gpt-4o-mini', 42, 0);
console.log('Cooldown cleared');

```

### Force Limit Re-Learning

Update the stored limit after receiving a new provider error:

```typescript
import { learnLimitFromError } from './ratelimit';

const err = { message: 'Limit 5000 requests per day' };
learnLimitFromError(42, err); // 42 = model_db_id

```

## Summary

- **FreeLLMAPI** uses a three-layer architecture: routing chain ordering, key selection with cooldown checks, and persistent rate limit tracking.
- **Inspect the chain** with `getOrderedFusionChain()` to verify models are available before troubleshooting keys.
- **Check penalties** via `getAllPenalties()` and **rate usage** via `getRateLimitStatus()` to identify blocked resources.
- **Verify cooldowns** using `isOnCooldown()` to confirm if keys are temporarily disabled.
- **Read diagnostics** by catching `RouteError` and inspecting the `diagnostics` array, which lists exact failure reasons for each model.
- **Learn new limits** with `learnLimitFromError()` when providers change their quotas or return limit errors.

## Frequently Asked Questions

### What does "All models exhausted" mean in FreeLLMAPI?

This error indicates that the routing chain iterated through every available model but failed to find a usable API key for any of them. According to the source code in [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts), this occurs when `selectKeyForModel` rejects all keys for every model due to cooldowns, rate limits, or missing capabilities. The `RouteError` thrown contains a `diagnostics` array that specifies exactly why each model was skipped.

### How do I check if a specific API key is on cooldown?

Import `isOnCooldown` from [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) and pass the platform string, model ID, and key ID. The function queries the SQLite-backed cooldown store (defined in the database layer) and returns a boolean indicating whether the key is currently blocked. This check happens automatically during routing, but you can call it manually to verify key health before submitting requests.

### Where does FreeLLMAPI store rate limit usage data?

Rate limit counters persist in SQLite tables defined in [`server/src/db/model-pricing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/db/model-pricing.ts), specifically `rate_limit_usage` and `rate_limit_cooldowns`. The [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) service manages sliding-window counters for RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), and TPD (tokens per day), updating these records after each request and checking them during the `selectKeyForModel` phase.

### How can I reset rate limit penalties for testing?

To clear penalties during development, import `setCooldown` from [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) and set the duration to `0` for the specific platform, model, and key ID. Alternatively, you can manually adjust the penalty counters in memory if you have access to the internal state, though the cooldown reset is the safer production-adjacent method. For penalty scores specifically, restart the server process to clear the in-memory penalty map maintained by `recordRateLimitHit`.