How to Troubleshoot "All Models Exhausted" Errors in FreeLLMAPI

The "All models exhausted" error occurs when FreeLLMAPI's router exhausts every available model and API key in the fallback chain due to rate limits, quota depletion, or configuration mismatches.

FreeLLMAPI aggregates multiple free-tier LLM providers behind a unified OpenAI-compatible API. When every configured key is temporarily unavailable, the router throws a RouteError with the message "All models exhausted" and returns HTTP 429. Understanding the five underlying services that track key health and quotas is essential to resolving this issue quickly.

Understanding the Routing Architecture

The error originates in server/src/services/router.ts at line 891, where the router fails to find any viable provider after walking the ordered fallback chain. Before throwing the error, the system checks each candidate against real-time constraints managed by supporting services.

Key Components That Trigger Exhaustion

• Router (router.ts): Executes the fallback chain logic. If no model passes constraints (context window, TPM limits), it throws the 429 error with the message "All models exhausted. Add more API keys or wait for rate limits to reset."
• Rate-Limit Ledger (ratelimit.ts): Tracks per-key RPM/RPD/TPM/TPD counters. A burst of requests can push every key over its per-minute quota, triggering cooldown periods that remove keys from rotation.
• Provider Quota (provider-quota.ts): Enforces daily token budgets per (provider, model, key) pair. When the daily budget is consumed (e.g., Gemini Pro's 1M tokens/month limit), the key is treated as exhausted until UTC midnight.
• Health Service (health.ts): Probes keys periodically; keys returning errors are marked error or rate_limited, removing them from the pool.
• Request-Retention (request-retention.ts): Enforces sticky sessions to avoid re-using a key that just failed. A key that returned a 429 may be skipped for the next few seconds, accelerating exhaustion.

Common Causes of "All Models Exhausted"

Free-Tier Quota Depletion

Free providers allocate limited tokens per day. Once the daily budget tracked in provider-quota.ts is spent, the router skips those keys until the UTC reset.

Per-Minute Rate Limits

A rapid burst of requests can exceed RPM or TPM limits tracked in ratelimit.ts, putting each key on a short cooldown and causing the router to exhaust all candidates quickly.

Provider Outages or Invalid Keys

The health service in health.ts marks keys as unhealthy or invalid if the provider is down or credentials are wrong, effectively removing them from the available pool.

Misconfigured Fallback Chains

If the first model in the chain cannot handle the request type (e.g., vision requests on text-only models), the router skips it. If all subsequent keys are also exhausted, the error triggers at line 891 of router.ts.

Context Window Mismatches

If the request's token count exceeds a model's context window or TPM limit (checked at lines 71-82 of router.ts), that model is skipped, potentially leaving no viable alternatives.

Diagnostic Steps

Check Server Logs

Look for RouteError: All models exhausted in the server logs. The stack trace reveals which constraint failed (quota, rate limit, or health check).

Inspect the Health Dashboard

Navigate to the Keys page in the dashboard. Keys marked rate_limited, error, or invalid are being skipped by the router according to the logic in health.ts.

Verify Daily Token Budgets

Open Models → Chat or Models → Embeddings to view remaining daily budgets for each provider. Free tiers typically show low remaining balances near UTC midnight.

Monitor Response Headers

Successful responses include the X-Routed-Via header indicating which provider served the request. Repeated identical values suggest you're hitting the same quota repeatedly before exhausting the chain.

Resolution Strategies

Add More API Keys

Increase redundancy by adding additional free-tier keys via the dashboard:

1. Open Keys → Add Provider (e.g., Google, OpenRouter).
2. Paste the API key and save.
3. Verify the key shows a green healthy status in the UI.

Reorder the Fallback Chain

Prioritize models with larger daily budgets:

1. Navigate to Models → Fallback Chain.
2. Drag models with higher quotas to the top.
3. The order persists in the SQLite fallback_config table.

Implement Client-Side Throttling

Insert delays between requests to stay under per-minute caps tracked in ratelimit.ts:

import time
from openai import OpenAI

client = OpenAI(base_url="http://localhost:3001/v1", 
                api_key="freellmapi-your-unified-key")

# Implement exponential backoff

for attempt in range(3):
    try:
        resp = client.chat.completions.create(
            model="auto",
            messages=[{"role": "user", "content": "Hello"}]
        )
        print("Routed via:", resp.headers.get("x-routed-via"))
        break
    except Exception as e:
        if "exhausted" in str(e):
            time.sleep(2 ** attempt)  # 1s, 2s, 4s

Code Examples

Triggering the Error via cURL

Test if your instance is exhausted:

curl -X POST http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role":"user","content":"Explain quantum tunnelling"}],
    "max_tokens": 2048
  }' -w "\nHTTP status: %{http_code}\n"

Expected output when exhausted:

{
  "error": {
    "message": "All models exhausted. Add more API keys or wait for rate limits to reset.",
    "type": "rate_limit_error"
  }
}

Checking Health Status

Verify which keys are rate-limited using the health endpoint:

curl http://localhost:3001/api/health

Look for keys with "status": "rate_limited" or "status": "error" in the JSON response to identify which providers are unavailable.

Handling Errors in Python

Gracefully catch the exhaustion error thrown by router.ts:

from openai import OpenAI, RateLimitError

client = OpenAI(base_url="http://localhost:3001/v1",
                api_key="freellmapi-your-unified-key")

try:
    resp = client.chat.completions.create(
        model="auto",
        messages=[{"role": "user", "content": "Tell me a joke"}],
    )
    print(resp.choices[0].message.content)
    print("Routed via:", resp.headers.get("x-routed-via"))
except RateLimitError:
    # Triggered when router.ts throws at line 891

    print("All free tiers exhausted. Add more keys or wait for UTC reset.")

Summary

• "All models exhausted" is a HTTP 429 error thrown by server/src/services/router.ts at line 891 when every API key is unavailable due to rate limits, quota exhaustion, or health checks.
• Root causes fall into five categories: daily quota depletion (provider-quota.ts), per-minute rate limits (ratelimit.ts), provider health issues (health.ts), sticky session exclusions (request-retention.ts), and configuration mismatches in the fallback chain.
• Immediate fixes include adding more free-tier keys via the Keys dashboard, reordering the fallback chain in Models → Fallback Chain, and implementing exponential backoff in clients.
• Diagnostic tools include server logs, the /api/health endpoint, and the X-Routed-Via response header.

Frequently Asked Questions

Why does the error say "All models exhausted" when I just added a new key?

The new key may be marked as unhealthy by health.ts due to invalid credentials, or it may already be rate-limited from previous usage. Check the Keys dashboard to confirm the status shows healthy and verify the key has remaining daily quota in Models → Chat.

How long should I wait after seeing this error before retrying?

Wait time depends on the limiting factor. For RPM/TPM limits enforced by ratelimit.ts, cooldowns typically last 1-60 seconds. For daily quotas tracked in provider-quota.ts, you must wait until UTC midnight when the ledger resets.

Can I prevent this error by using only paid API keys?

FreeLLMAPI supports paid keys, but you must configure them with higher rate limits and place them at the top of your fallback chain. Paid keys bypass free-tier quotas but are still subject to their own RPM/TPM limits tracked in ratelimit.ts and health checks in health.ts.

What does the X-Routed-Via header tell me about exhaustion patterns?

This header reveals which provider/model served the request. If you see the same provider repeatedly before hitting the exhaustion error, it indicates that provider's quota is depleting fastest. Use this data to reorder your fallback chain and diversify across providers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}