# How to Implement Custom Error Classification for AI Provider Failures in Open-Notebook

> Implement custom error classification for AI provider failures in Open Notebook. Learn how to map raw exceptions to structured errors for consistent user messages across workflows.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-22

---

**Open-Notebook centralizes AI provider error handling through a stateless classifier in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py) that maps raw exceptions to structured `OpenNotebookError` subclasses using keyword matching, enabling consistent user-facing messages across all graph workflows.**

Open-Notebook provides a robust mechanism for handling failures from diverse AI providers like OpenAI, Anthropic, and Google through a centralized error classification system. By implementing custom error classification for AI provider failures, you ensure that obscure provider-specific exceptions transform into clear, actionable feedback while maintaining detailed logs for diagnostics. The architecture leverages a data-driven rule table that requires no changes to downstream call sites when extending support for new failure modes.

## Understanding the Error Classification Architecture

### The Central Classification Module

The error classification logic resides in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py), which exports the `classify_error()` function. This module maintains a private `_CLASSIFICATION_RULES` list containing tuples of `(keywords, exception_class, user_message)` that define how raw provider exceptions map to Open-Notebook's structured error hierarchy.

The classifier is **stateless and safe for async contexts**, inspecting only the exception object without maintaining internal state. When called, it constructs a lower-cased string combining the exception's type name and message content, then checks this against the keyword patterns defined in the rule table.

### The Three-Step Classification Pattern

The workflow follows a predictable three-step pattern used throughout the graph layer:

1. **Collect raw exceptions** – Workflow modules like [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py), [`source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/source_chat.py), and [`ask.py`](https://github.com/lfnovo/open-notebook/blob/main/ask.py) catch exceptions from LangChain or Esperanto calls and forward them to `classify_error`.

2. **Match against classification rules** – The utility iterates through `_CLASSIFICATION_RULES`, checking if any keyword in the rule appears within the lower-cased exception string.

3. **Return a predictable Open-Notebook error** – The function returns a concrete subclass of `OpenNotebookError` (e.g., `AuthenticationError`, `RateLimitError`, `NetworkError`) together with a concise user-friendly message. Unrecognized errors are logged and wrapped as `ExternalServiceError`.

## Step-by-Step Guide to Adding Custom Classifications

### Step 1: Identify the New Failure Mode

Determine the distinctive phrases that appear in the raw exception text when the provider encounters the specific failure. The classifier converts the exception to a lower-cased string for matching, so you should identify unique keywords that appear in both the exception type name and message.

```python

# Example: Extracting the search string

exception_str = str(exc).lower()

```

### Step 2: Choose or Create an Exception Class

Select an appropriate subclass of `OpenNotebookError` from [`open_notebook/exceptions.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/exceptions.py), or create a new one for provider-specific errors. Custom exceptions should inherit from the base `OpenNotebookError` class to ensure consistent handling by the API routers.

```python

# In open_notebook/exceptions.py

class ProviderUnavailableError(OpenNotebookError):
    """Raised when the AI provider cannot allocate necessary resources."""
    pass

```

### Step 3: Add a Classification Rule

Append a new tuple to `_CLASSIFICATION_RULES` in [`error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/error_classifier.py). The tuple format is `(keywords_list, ExceptionClass, user_message)`, where `user_message` can be `None` to forward the original exception text.

```python

# At the bottom of _CLASSIFICATION_RULES in error_classifier.py

(
    ["gpu unavailable", "no gpu"],  # keywords to match

    ProviderUnavailableError,       # custom OpenNotebookError subclass

    "The provider cannot allocate GPU resources. Try again later.",  # friendly message

),

```

### Step 4: Validate with Unit Tests

Create a test that raises a mock exception containing one of your keywords and asserts that `classify_error` returns the expected class and message. The [`tests/test_embedding.py`](https://github.com/lfnovo/open-notebook/blob/main/tests/test_embedding.py) file provides example patterns for verifying error classification behavior.

```python
from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import ProviderUnavailableError

def test_gpu_unavailable_classification():
    class DummyExc(Exception):
        pass

    exc = DummyExc("GPU unavailable – please retry")
    exc_class, msg = classify_error(exc)

    assert exc_class is ProviderUnavailableError
    assert "GPU resources" in msg

```

### Step 5: Deploy

No further changes are required to the graph modules or API routers. Because `classify_error` is imported throughout the graph layer ([`chat.py`](https://github.com/lfnovo/open-notebook/blob/main/chat.py), [`ask.py`](https://github.com/lfnovo/open-notebook/blob/main/ask.py), [`source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/source_chat.py), [`transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/transformation.py)), the updated rule becomes instantly effective for all downstream endpoints including Chat, Ask, Source-Chat, and Search operations.

## Code Implementation Examples

### Calling the Classifier Directly

When implementing new provider integrations, wrap raw calls with the classifier to ensure consistent error handling:

```python
from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import OpenNotebookError

try:
    # Some provider call that may raise a raw exception

    result = provider.generate(...)
except Exception as exc:
    exc_class, user_msg = classify_error(exc)
    # Re-raise a uniform Open-Notebook error

    raise exc_class(user_msg) from exc

```

### Integration Points Across the Codebase

The classifier is imported and utilized across multiple critical graph modules:

- [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) – Handles conversation flow errors
- [`open_notebook/graphs/source_chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source_chat.py) – Manages source-specific chat failures  
- [`open_notebook/graphs/ask.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/ask.py) – Processes question-answering errors
- [`open_notebook/graphs/transformation.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/transformation.py) – Catches transformation pipeline failures

Because these modules rely on the centralized `classify_error` function, any new rule you add automatically extends error handling coverage to all existing endpoints without modifying individual call sites.

## Summary

- **Centralized architecture**: All AI provider error classification flows through [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py), ensuring consistent behavior across Chat, Ask, and Search workflows.
- **Data-driven extensibility**: Adding support for new failure modes requires only appending a tuple to `_CLASSIFICATION_RULES`, with no changes needed to the graph layer modules.
- **Structured error hierarchy**: Raw exceptions transform into specific `OpenNotebookError` subclasses (like `AuthenticationError` or `RateLimitError`) with user-friendly messages.
- **Stateless design**: The classifier safely operates in async contexts throughout the application, inspecting exception text without maintaining internal state.
- **Comprehensive coverage**: Unrecognized errors automatically fall back to `ExternalServiceError`, ensuring no raw provider exceptions leak to users unhandled.

## Frequently Asked Questions

### How does the classifier handle exceptions that don't match any known rules?

When `classify_error` encounters an exception that matches no keywords in `_CLASSIFICATION_RULES`, it logs the original exception for diagnostic purposes and returns `ExternalServiceError` along with the original exception message. This ensures users receive a generic but safe error notification while developers retain access to debugging information.

### Can I use regular expressions instead of simple keyword matching in classification rules?

The current implementation in [`open_notebook/utils/error_classifier.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/error_classifier.py) uses simple substring keyword matching against the lower-cased exception string. While the source code does not natively support regex patterns, you can implement complex matching logic by subclassing the exception handling in your specific graph module or by proposing a modification to the `_CLASSIFICATION_RULES` structure to support callable matchers.

### Where should I define custom exception classes for new AI provider failures?

Define all custom exception classes in [`open_notebook/exceptions.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/exceptions.py) to maintain a clean separation of concerns. These classes should inherit from `OpenNotebookError` to ensure the API routers and frontend components recognize them as application-specific errors rather than unhandled system exceptions.

### What is the performance impact of the error classification system?

The classifier introduces minimal overhead because it operates only when exceptions occur and performs simple string operations (lower-casing and substring searches) against a static rule table. Because the utility is stateless and does not perform I/O operations, it adds negligible latency to the error handling path even when processing high volumes of failed requests.