# How the Open-Notebook API Handles Rate Limiting from AI Providers

> Learn how the Open-Notebook API expertly manages AI provider rate limits, converting errors into structured exceptions and HTTP 429 responses with actionable retry messages.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-29

---

**When AI providers return rate-limit errors, the Open-Notebook API converts them into structured `RateLimitError` exceptions and returns HTTP 429 responses with clear retry messages.**

The Open-Notebook project (`lfnovo/open-notebook`) provides a unified interface for interacting with multiple large language model providers. When these providers enforce rate limits, the API implements a centralized error classification system that standardizes how rate limiting is surfaced to clients through consistent HTTP responses.

## Error Classification Architecture

The architecture separates provider-specific error detection from HTTP response generation. This ensures that adding new AI providers only requires updating the classification logic in one location.

### Detecting Provider Rate Limits

When the API calls an AI model through LangChain graphs, the `classify_error` utility in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py) scans exception messages for rate-limit indicators. The classifier checks for keywords including `"rate limit"`, `"429"`, and `"too many requests"` to identify when a provider has throttled the request.

If the classifier detects a rate-limit pattern, it maps the raw provider exception to the custom `RateLimitError` class defined in [`open_notebook/exceptions.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/exceptions.py). This transformation occurs before the error propagates to the API layer, ensuring that all provider-specific details are abstracted into the Open-Notebook error hierarchy.

### The classify_error Helper Function

The `classify_error` function serves as the central registry for error translation. It accepts any exception raised by AI providers and returns a tuple containing the appropriate error class and a user-friendly message. For rate limiting, this returns `RateLimitError` with a message instructing users to retry later.

This centralized approach means that updates to provider error formats or the addition of new AI services only require modifications to [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py), leaving the rest of the error handling pipeline unchanged.

## From Exception to HTTP 429 Response

The error handling pipeline operates across two layers: the graph execution layer where models are invoked, and the FastAPI application layer where HTTP responses are generated.

### Graph-Level Exception Handling

In [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py), model invocations are wrapped in try-except blocks that capture any provider exceptions. The code passes the caught exception to `classify_error` and immediately re-raises it as the classified domain error:

```python

# From open_notebook/graphs/chat.py

try:
    ai_message = model.invoke(payload)
except Exception as e:
    error_class, user_message = classify_error(e)
    raise error_class(user_message) from e

```

This pattern ensures that regardless of which AI provider (OpenAI, Anthropic, or others) generated the original error, the calling code receives a consistent `RateLimitError` that the API layer knows how to convert into an HTTP response.

### FastAPI Global Exception Handler

The [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) file registers a global exception handler for `RateLimitError` that converts the domain exception into a standard HTTP 429 response:

```python

# From api/main.py

@app.exception_handler(RateLimitError)
async def rate_limit_error_handler(request: Request, exc: RateLimitError):
    return JSONResponse(
        status_code=429,
        content={"detail": str(exc)},
        headers=_cors_headers(request),
    )

```

The handler includes CORS headers in the response, ensuring that frontend applications can read the error details without cross-origin restrictions. This means every endpoint that ultimately calls an LLM—whether chat, ask, or transformation endpoints—returns rate-limit failures in the same format.

## Implementation Example

When a client calls the chat endpoint and the underlying provider enforces a rate limit, the API returns a structured response:

```python
import httpx

async def ask_chat():
    async with httpx.AsyncClient() as client:
        resp = await client.post(
            "http://localhost:5055/api/chat",
            json={"messages": [{"role": "user", "content": "Hello"}]},
        )
        # Rate limit response:

        # status_code == 429

        # json() == {"detail": "Rate limit exceeded. Please wait a moment and try again."}

        return resp

```

The response body contains a JSON payload with a `detail` key explaining the error, while the 429 status code allows client applications to implement retry logic with exponential backoff.

## Benefits of Centralized Rate Limit Handling

This architecture provides several advantages for applications consuming the Open-Notebook API:

- **Consistent Client Experience**: Frontend applications always receive HTTP 429 responses with clear messages, enabling them to display appropriate "retry later" prompts to users.
- **Maintainable Provider Integration**: All provider-specific error parsing lives in `classify_error`, making it simple to add new AI services or update keyword matching as provider error formats evolve.
- **Observable Error Patterns**: The classifier logs warnings for unrecognized errors, helping developers identify new rate-limit formats that may require additional classification rules.

## Summary

- The Open-Notebook API handles rate limiting through a centralized classification system in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py).
- The `classify_error` function scans provider exceptions for keywords like "rate limit", "429", and "too many requests" to create `RateLimitError` instances.
- Graph implementations in files like [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) wrap model calls and propagate classified errors to the FastAPI layer.
- The global exception handler in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) converts `RateLimitError` into HTTP 429 responses with CORS headers and user-friendly detail messages.
- All LLM endpoints inherit this behavior, providing a uniform API contract for rate-limit handling across the entire application.

## Frequently Asked Questions

### What HTTP status code does the Open-Notebook API return for rate limits?

The API returns HTTP status code **429 Too Many Requests** when an AI provider enforces a rate limit. This standard status code allows HTTP clients to automatically detect throttling conditions and implement appropriate retry strategies.

### How does the API distinguish between different AI provider error formats?

The `classify_error` function in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py) uses keyword matching to identify rate-limit errors across different provider formats. It scans exception messages for terms like "rate limit", "429", and "too many requests", then maps matching exceptions to the `RateLimitError` class regardless of which provider generated the original error.

### Can custom rate-limiting logic be added for specific providers?

Yes, the centralized classification system makes it straightforward to add provider-specific logic. By modifying the `classify_error` function in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py), developers can add new keyword patterns or provider-specific exception types while maintaining the same `RateLimitError` output and HTTP 429 response behavior throughout the API.

### Where is the error message formatting handled when rate limits occur?

Error message formatting occurs in the `classify_error` utility, which returns a user-friendly message alongside the error class. This message is passed through to the HTTP response in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py), ensuring that clients receive consistent, human-readable descriptions like "Rate limit exceeded. Please wait a moment and try again."