# How to Troubleshoot "All Models Exhausted" Errors in FreeLLMAPI

> Resolve 'All models exhausted' errors in FreeLLMAPI. Learn to fix rate limits, quota issues, and configuration problems for seamless model access. Get your FreeLLMAPI running smoothly!

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-23

---

**The "All models exhausted" error occurs when FreeLLMAPI's router exhausts every available model and API key in the fallback chain due to rate limits, quota depletion, or configuration mismatches.**

FreeLLMAPI aggregates multiple free-tier LLM providers behind a unified OpenAI-compatible API. When every configured key is temporarily unavailable, the router throws a `RouteError` with the message "All models exhausted" and returns HTTP 429. Understanding the five underlying services that track key health and quotas is essential to resolving this issue quickly.

## Understanding the Routing Architecture

The error originates in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) at line 891, where the router fails to find any viable provider after walking the ordered fallback chain. Before throwing the error, the system checks each candidate against real-time constraints managed by supporting services.

### Key Components That Trigger Exhaustion

- **Router ([`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts))**: Executes the fallback chain logic. If no model passes constraints (context window, TPM limits), it throws the 429 error with the message "All models exhausted. Add more API keys or wait for rate limits to reset."
- **Rate-Limit Ledger ([`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts))**: Tracks per-key RPM/RPD/TPM/TPD counters. A burst of requests can push every key over its per-minute quota, triggering cooldown periods that remove keys from rotation.
- **Provider Quota ([`provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/provider-quota.ts))**: Enforces daily token budgets per `(provider, model, key)` pair. When the daily budget is consumed (e.g., Gemini Pro's 1M tokens/month limit), the key is treated as exhausted until UTC midnight.
- **Health Service ([`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts))**: Probes keys periodically; keys returning errors are marked `error` or `rate_limited`, removing them from the pool.
- **Request-Retention ([`request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/request-retention.ts))**: Enforces sticky sessions to avoid re-using a key that just failed. A key that returned a 429 may be skipped for the next few seconds, accelerating exhaustion.

## Common Causes of "All Models Exhausted"

### Free-Tier Quota Depletion

Free providers allocate limited tokens per day. Once the daily budget tracked in [`provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/provider-quota.ts) is spent, the router skips those keys until the UTC reset.

### Per-Minute Rate Limits

A rapid burst of requests can exceed RPM or TPM limits tracked in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts), putting each key on a short cooldown and causing the router to exhaust all candidates quickly.

### Provider Outages or Invalid Keys

The health service in [`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts) marks keys as `unhealthy` or `invalid` if the provider is down or credentials are wrong, effectively removing them from the available pool.

### Misconfigured Fallback Chains

If the first model in the chain cannot handle the request type (e.g., vision requests on text-only models), the router skips it. If all subsequent keys are also exhausted, the error triggers at line 891 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts).

### Context Window Mismatches

If the request's token count exceeds a model's context window or TPM limit (checked at lines 71-82 of [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts)), that model is skipped, potentially leaving no viable alternatives.

## Diagnostic Steps

### Check Server Logs

Look for `RouteError: All models exhausted` in the server logs. The stack trace reveals which constraint failed (quota, rate limit, or health check).

### Inspect the Health Dashboard

Navigate to the **Keys** page in the dashboard. Keys marked `rate_limited`, `error`, or `invalid` are being skipped by the router according to the logic in [`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts).

### Verify Daily Token Budgets

Open **Models → Chat** or **Models → Embeddings** to view remaining daily budgets for each provider. Free tiers typically show low remaining balances near UTC midnight.

### Monitor Response Headers

Successful responses include the `X-Routed-Via` header indicating which provider served the request. Repeated identical values suggest you're hitting the same quota repeatedly before exhausting the chain.

## Resolution Strategies

### Add More API Keys

Increase redundancy by adding additional free-tier keys via the dashboard:

1. Open **Keys** → **Add Provider** (e.g., Google, OpenRouter).
2. Paste the API key and save.
3. Verify the key shows a green `healthy` status in the UI.

### Reorder the Fallback Chain

Prioritize models with larger daily budgets:

1. Navigate to **Models → Fallback Chain**.
2. Drag models with higher quotas to the top.
3. The order persists in the SQLite `fallback_config` table.

### Implement Client-Side Throttling

Insert delays between requests to stay under per-minute caps tracked in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts):

```python
import time
from openai import OpenAI

client = OpenAI(base_url="http://localhost:3001/v1", 
                api_key="freellmapi-your-unified-key")

# Implement exponential backoff

for attempt in range(3):
    try:
        resp = client.chat.completions.create(
            model="auto",
            messages=[{"role": "user", "content": "Hello"}]
        )
        print("Routed via:", resp.headers.get("x-routed-via"))
        break
    except Exception as e:
        if "exhausted" in str(e):
            time.sleep(2 ** attempt)  # 1s, 2s, 4s

```

## Code Examples

### Triggering the Error via cURL

Test if your instance is exhausted:

```bash
curl -X POST http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role":"user","content":"Explain quantum tunnelling"}],
    "max_tokens": 2048
  }' -w "\nHTTP status: %{http_code}\n"

```

Expected output when exhausted:

```json
{
  "error": {
    "message": "All models exhausted. Add more API keys or wait for rate limits to reset.",
    "type": "rate_limit_error"
  }
}

```

### Checking Health Status

Verify which keys are rate-limited using the health endpoint:

```bash
curl http://localhost:3001/api/health

```

Look for keys with `"status": "rate_limited"` or `"status": "error"` in the JSON response to identify which providers are unavailable.

### Handling Errors in Python

Gracefully catch the exhaustion error thrown by [`router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/router.ts):

```python
from openai import OpenAI, RateLimitError

client = OpenAI(base_url="http://localhost:3001/v1",
                api_key="freellmapi-your-unified-key")

try:
    resp = client.chat.completions.create(
        model="auto",
        messages=[{"role": "user", "content": "Tell me a joke"}],
    )
    print(resp.choices[0].message.content)
    print("Routed via:", resp.headers.get("x-routed-via"))
except RateLimitError:
    # Triggered when router.ts throws at line 891

    print("All free tiers exhausted. Add more keys or wait for UTC reset.")

```

## Summary

- **"All models exhausted"** is a HTTP 429 error thrown by [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) at line 891 when every API key is unavailable due to rate limits, quota exhaustion, or health checks.
- **Root causes** fall into five categories: daily quota depletion ([`provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/provider-quota.ts)), per-minute rate limits ([`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts)), provider health issues ([`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts)), sticky session exclusions ([`request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/request-retention.ts)), and configuration mismatches in the fallback chain.
- **Immediate fixes** include adding more free-tier keys via the **Keys** dashboard, reordering the fallback chain in **Models → Fallback Chain**, and implementing exponential backoff in clients.
- **Diagnostic tools** include server logs, the `/api/health` endpoint, and the `X-Routed-Via` response header.

## Frequently Asked Questions

### Why does the error say "All models exhausted" when I just added a new key?

The new key may be marked as unhealthy by [`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts) due to invalid credentials, or it may already be rate-limited from previous usage. Check the **Keys** dashboard to confirm the status shows `healthy` and verify the key has remaining daily quota in **Models → Chat**.

### How long should I wait after seeing this error before retrying?

Wait time depends on the limiting factor. For RPM/TPM limits enforced by [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts), cooldowns typically last 1-60 seconds. For daily quotas tracked in [`provider-quota.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/provider-quota.ts), you must wait until UTC midnight when the ledger resets.

### Can I prevent this error by using only paid API keys?

FreeLLMAPI supports paid keys, but you must configure them with higher rate limits and place them at the top of your fallback chain. Paid keys bypass free-tier quotas but are still subject to their own RPM/TPM limits tracked in [`ratelimit.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/ratelimit.ts) and health checks in [`health.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/health.ts).

### What does the `X-Routed-Via` header tell me about exhaustion patterns?

This header reveals which provider/model served the request. If you see the same provider repeatedly before hitting the exhaustion error, it indicates that provider's quota is depleting fastest. Use this data to reorder your fallback chain and diversify across providers.