# How to Handle Rate Limiting Errors from AI Providers with Retry Logic in Open Notebook

> Open Notebook's retry logic handles AI provider rate limit errors, ensuring robust API communication with exponential backoff. Detects and converts errors to RateLimitError. Learn more.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-07-05

---

**Open Notebook automatically detects rate-limit responses from AI providers, converts them to structured `RateLimitError` exceptions, and implements configurable retry logic with exponential backoff to ensure robust API communication.**

When building AI-powered applications with Open Notebook, encountering 429 "Too Many Requests" errors from providers like OpenAI or Anthropic is inevitable. The application uses a FastAPI backend that communicates with AI services through the **Esperanto** library, implementing a sophisticated error classification and retry system. This architecture ensures that transient rate limits don't disrupt user workflows while providing clear feedback when persistent limits are reached.

## Understanding the Error Classification Pipeline

All provider-specific exceptions flow through a centralized classification system before reaching your application logic. This design allows Open Notebook to normalize different provider error formats into consistent exception types.

### Mapping Provider Exceptions to RateLimitError

In [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py), the `classify_error` function contains a rule set that identifies rate-limit patterns across different AI providers:

```python

# open_notebook/utils/error_classifier.py

(
    ["rate limit", "rate_limit", "429", "too many requests", "quota exceeded"],
    RateLimitError,
    "Rate limit exceeded. Please wait a moment and try again.",
),

```

When the exception text matches any of these keywords, the function returns a `RateLimitError` instance along with a user-friendly message. This classification runs throughout the graph layers in `open_notebook/graphs/*`, ensuring consistent error handling regardless of which provider generated the original exception.

## The RateLimitError Exception Type

The concrete exception class lives in [`open_notebook/exceptions.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/exceptions.py), providing a distinct type for catching rate-limit specific failures:

```python

# open_notebook/exceptions.py

class RateLimitError(OpenNotebookError):
    """Raised when a rate limit is exceeded."""
    pass

```

Inheriting from `OpenNotebookError` allows the application to distinguish rate-limit errors from other AI provider failures, enabling targeted retry strategies while maintaining a clear exception hierarchy.

## HTTP Response Handling in FastAPI

Open Notebook registers a dedicated exception handler in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) to ensure rate-limit errors translate to proper HTTP responses:

```python

# api/main.py

@app.exception_handler(RateLimitError)
async def rate_limit_error_handler(request: Request, exc: RateLimitError):
    return JSONResponse(
        status_code=429,
        content={"detail": str(exc)},
        headers=_cors_headers(request),
    )

```

This handler automatically converts any uncaught `RateLimitError` into a **429 Too Many Requests** response, preserving CORS headers for frontend consumption. The result is a clean API contract that informs clients exactly when they need to back off.

## Built-in Retry Logic for Embeddings

Open Notebook implements generic retry logic for transient failures in the embedding pipeline, which automatically catches and retries rate-limit errors.

### Configuration Constants

The retry behavior is defined in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py):

```python

# open_notebook/utils/embedding.py

EMBEDDING_MAX_RETRIES = 3          # how many attempts per batch

EMBEDDING_RETRY_DELAY = 2          # seconds to wait before a retry

```

### The Retry Loop Implementation

Each batch of texts processes through a retry loop that catches `RateLimitError` (classified from provider exceptions) and waits before retrying:

```python

# open_notebook/utils/embedding.py

for attempt in range(1, EMBEDDING_MAX_RETRIES + 1):
    try:
        batch_embeddings = await embedding_model.aembed(batch)
        all_embeddings.extend(batch_embeddings)
        break
    except Exception as e:
        # Log and optionally back-off before next attempt

        if attempt < EMBEDDING_MAX_RETRIES:
            logger.debug(
                f"Embedding batch {batch_idx+1}/{total_batches} attempt {attempt} failed: {e}. "
                "Retrying..."
            )
            await asyncio.sleep(EMBEDDING_RETRY_DELAY)
        else:
            raise RuntimeError(...)

```

Because `classify_error` converts low-level provider exceptions into `RateLimitError` before they reach this loop, the embedding pipeline automatically handles rate-limit hits from any provider without provider-specific code.

## Implementing Custom Retry Logic for LLM Calls

For operations outside the embedding pipeline, you can implement custom retry logic with exponential backoff using a pattern similar to the embedding code:

```python
import asyncio
from open_notebook.exceptions import RateLimitError
from loguru import logger

async def call_with_rate_limit_retry(coro, *, max_retries=5, base_delay=1.0):
    """Execute `coro` (an awaitable) and retry on RateLimitError."""
    for attempt in range(1, max_retries + 1):
        try:
            return await coro()
        except RateLimitError as exc:
            if attempt == max_retries:
                logger.error("Rate‑limit retries exhausted")
                raise
            delay = base_delay * (2 ** (attempt - 1))   # exponential back‑off

            logger.warning(
                f"Rate limit hit (attempt {attempt}/{max_retries}); sleeping {delay}s"
            )
            await asyncio.sleep(delay)

```

### Example Usage with Language Models

Apply this wrapper to LLM calls using the model manager from [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py):

```python
from open_notebook.ai.models import model_manager
from open_notebook.utils.error_classifier import classify_error

async def generate_chat_response(messages):
    async def _invoke():
        lm = await model_manager.get_language_model()
        return await lm.chat(messages)

    try:
        return await call_with_rate_limit_retry(_invoke)
    except Exception as e:
        # Convert any remaining exception into a user‑friendly message

        exc_class, user_msg = classify_error(e)
        raise exc_class(user_msg) from e

```

This pattern ensures that temporary rate limits trigger automatic retries with exponential backoff, while persistent failures surface as user-friendly errors after exhaustion.

## Configuration and Tuning

Open Notebook exposes several configuration points for adjusting retry behavior:

- **Environment Variables**: Tune the embedding retry behavior without code changes:
  - `OPEN_NOTEBOOK_EMBEDDING_BATCH_SIZE` – controls the size of each batch sent to the provider
  - `EMBEDDING_MAX_RETRIES` and `EMBEDDING_RETRY_DELAY` – defined in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py) and editable if you need different defaults

- **Provider-Specific Headers**: Some providers return a `Retry-After` header in their 429 responses. While the generic retry loop uses fixed delays, you can extend `call_with_rate_limit_retry` to parse this header and pass dynamic delays for more accurate backoff timing.

## Summary

- **Error Classification**: The `classify_error` function in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py) maps provider-specific rate-limit messages to the standard `RateLimitError` exception.
- **Exception Handling**: `RateLimitError` in [`open_notebook/exceptions.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/exceptions.py) provides a distinct type for catching these failures, while [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) converts them to HTTP 429 responses.
- **Automatic Retries**: The embedding pipeline in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py) implements built-in retry logic with configurable `EMBEDDING_MAX_RETRIES` and `EMBEDDING_RETRY_DELAY` constants.
- **Custom Implementation**: Use the `call_with_rate_limit_retry` pattern with exponential backoff for LLM operations and other AI provider calls.
- **User Experience**: The system surfaces clear "Rate limit exceeded" messages while automatically backing off and retrying, minimizing disruption during temporary provider throttling.

## Frequently Asked Questions

### How does Open Notebook detect rate limiting errors from different AI providers?

Open Notebook uses the `classify_error` function in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py) to scan exception messages for keywords like "rate limit", "429", "too many requests", and "quota exceeded". When any provider returns an error containing these strings, the function normalizes it into a `RateLimitError` with a consistent user-friendly message, regardless of whether the underlying provider is OpenAI, Anthropic, or a local Ollama instance.

### What is the default retry behavior for embedding operations?

The embedding pipeline in [`open_notebook/utils/embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/embedding.py) automatically retries failed batches up to **3 times** (`EMBEDDING_MAX_RETRIES = 3`) with a fixed delay of **2 seconds** (`EMBEDDING_RETRY_DELAY = 2`) between attempts. This catches rate-limit errors after they've been classified by the error classifier, ensuring that brief provider throttling doesn't fail the entire embedding job.

### Can I customize the retry logic for specific AI providers?

Yes. While Open Notebook provides built-in retry logic for embeddings, you can implement custom retry strategies for LLM calls or other operations using the `call_with_rate_limit_retry` pattern shown in the implementation examples. This allows you to adjust `max_retries`, implement exponential backoff instead of fixed delays, or read provider-specific `Retry-After` headers to customize backoff timing per provider.

### What HTTP status code does Open Notebook return when rate limits are exceeded?

Open Notebook returns **HTTP 429 Too Many Requests** when rate-limit errors exhaust all retries or occur in contexts without retry logic. The FastAPI exception handler in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) catches `RateLimitError` instances and returns a JSON response with status code 429 and a detail message explaining that the rate limit was exceeded, allowing frontend clients to implement their own backoff strategies.