# What Happens When All Providers in the FreeLLMAPI Fallback Chain Fail?

> Learn what happens when all providers in the FreeLLMAPI fallback chain fail. Discover router responses, error codes, and how to handle these failures effectively.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-06-24

---

**When every provider in the FreeLLMAPI fallback chain fails, the router returns an HTTP 502 Bad Gateway with an `X-Fallback-Attempts` header and a JSON error body containing the code `"fallback_exhausted"`.**

FreeLLMAPI is an open-source unified API gateway that aggregates multiple LLM providers into a single endpoint. When all configured providers in the fallback chain are exhausted due to rate limits, timeouts, or server errors, the system implements a structured failure protocol defined in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) and [`server/src/lib/error-classify.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/error-classify.ts).

## How the Fallback Chain Exhaustion Occurs

The FreeLLMAPI router attempts each enabled provider sequentially until it either succeeds or exhausts the chain. For every provider attempt, the system performs three validation steps:

1. **Health Check**: Verifies the API key is healthy and under rate limits.
2. **SDK Execution**: Calls the provider's native SDK.
3. **Error Classification**: If the provider returns **429 (rate-limited)**, any **5xx** error, or a timeout, the router **cool-downs** that key, records the failure, and advances to the next provider.

This process continues for up to **20 attempts** (configurable) before the chain is considered exhausted.

## The Exhaustion Response Protocol

When the router determines that every available provider has failed, it stops retrying and returns a standardized error response to the client.

### HTTP Status and Headers

The response includes:
- **Status Code**: `502 Bad Gateway` (or `422` for malformed requests)
- **Header**: `X-Fallback-Attempts: N` where *N* represents the number of providers tried before giving up

### Error Body Structure

The JSON payload follows this exact schema:

```json
{
  "error": {
    "message": "All providers in the fallback chain failed",
    "code": "fallback_exhausted"
  }
}

```

This error structure is generated in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) and finalized through [`server/src/lib/error-classify.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/error-classify.ts), which converts provider-level failures into router-level HTTP codes.

## Handling Fallback Exhaustion in Client Code

You can detect and handle this condition using any OpenAI-compatible client by checking for the 502 status code.

### Python Client Example

```python
from openai import OpenAI, APIError

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key"
)

try:
    resp = client.chat.completions.create(
        model="auto",
        messages=[{"role": "user", "content": "Explain quantum tunnelling"}],
    )
    print(resp.choices[0].message.content)
except APIError as exc:
    if exc.status_code == 502:
        attempts = exc.headers.get("x-fallback-attempts", "unknown")
        print(f"All providers exhausted after {attempts} attempts")
    else:
        raise

```

### cURL Request and Response

```bash
curl -i http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "auto",
        "messages": [{"role": "user", "content": "Give me a poem about clouds"}]
      }'

```

When the fallback chain is fully exhausted, the response appears as:

```

HTTP/1.1 502 Bad Gateway
X-Fallback-Attempts: 20
Content-Type: application/json

{
  "error": {
    "message": "All providers in the fallback chain failed",
    "code": "fallback_exhausted"
  }
}

```

### Node.js Detection

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:3001/v1",
  apiKey: "freellmapi-your-unified-key",
});

try {
  const result = await client.chat.completions.create({
    model: "auto",
    messages: [{ role: "user", content: "What is the capital of Estonia?" }],
  });
  console.log(result.choices[0].message.content);
} catch (err) {
  if (err.response?.status === 502) {
    const attempts = err.response.headers["x-fallback-attempts"];
    console.warn(`Fallback chain exhausted after ${attempts} provider attempts`);
  } else {
    throw err;
  }
}

```

## Implementation Details in the Source Code

The fallback exhaustion logic is distributed across three key files in the tashfeenahmed/freellmapi repository:

**[`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)**
Contains the core routing loop that iterates through providers and generates the 502 response when the fallback list is exhausted.

**[`server/src/lib/error-classify.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/error-classify.ts)**
Transforms provider-specific errors (timeouts, rate limits, 5xx responses) into standardized router-level HTTP codes, including the 502 status for exhausted fallback chains.

**[`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts)**
Implements "wave" style fallback for tool-heavy requests, which ultimately delegates to the same 502 exhaustion path when every provider in the wave configuration fails.

## Summary

- **FreeLLMAPI** attempts up to 20 provider calls sequentially before declaring the fallback chain exhausted.
- **HTTP 502 Bad Gateway** is returned when all providers fail, accompanied by the `X-Fallback-Attempts` header showing the attempt count.
- **Error classification** occurs in [`server/src/lib/error-classify.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/error-classify.ts), while the routing logic resides in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).
- **Client applications** should inspect for status code 502 and the `fallback_exhausted` error code to handle complete provider exhaustion.
- **Tool-heavy requests** use the same exhaustion protocol through [`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts) even when processed in "waves."

## Frequently Asked Questions

### What is the maximum number of fallback attempts in FreeLLMAPI?

FreeLLMAPI attempts up to **20 providers** by default before exhausting the fallback chain, though this value is configurable in the router settings. The attempt count is communicated to clients via the `X-Fallback-Attempts` header in the final 502 response.

### How does FreeLLMAPI classify provider errors that trigger a fallback?

According to [`server/src/lib/error-classify.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/error-classify.ts), the router triggers a fallback and cool-down when a provider returns **HTTP 429 (rate-limited)**, any **5xx server error**, or experiences a **timeout**. These errors cause the router to mark the key as unhealthy and advance to the next provider in the chain.

### Can I distinguish between a single provider failure and a complete fallback chain failure?

Yes. Single provider failures are handled internally by the router and trigger automatic retries. Only when the entire chain is exhausted does FreeLLMAPI return **HTTP 502 Bad Gateway** with the error code `"fallback_exhausted"`. Successful requests return standard 200 OK responses, while malformed requests return 422.

### What happens to tool-heavy requests when all providers fail?

As implemented in [`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts), tool-heavy requests use a "wave" style fallback that attempts multiple providers in parallel groups. If every provider in every wave fails, the system falls back to the same **502 Bad Gateway** response with the `fallback_exhausted` error code, identical to standard sequential fallback exhaustion.