How to Implement Custom Error Classification for AI Provider Failures in Open-Notebook

Open-Notebook centralizes AI provider error handling through a stateless classifier in open_notebook/utils/error_classifier.py that maps raw exceptions to structured OpenNotebookError subclasses using keyword matching, enabling consistent user-facing messages across all graph workflows.

Open-Notebook provides a robust mechanism for handling failures from diverse AI providers like OpenAI, Anthropic, and Google through a centralized error classification system. By implementing custom error classification for AI provider failures, you ensure that obscure provider-specific exceptions transform into clear, actionable feedback while maintaining detailed logs for diagnostics. The architecture leverages a data-driven rule table that requires no changes to downstream call sites when extending support for new failure modes.

Understanding the Error Classification Architecture

The Central Classification Module

The error classification logic resides in open_notebook/utils/error_classifier.py, which exports the classify_error() function. This module maintains a private _CLASSIFICATION_RULES list containing tuples of (keywords, exception_class, user_message) that define how raw provider exceptions map to Open-Notebook's structured error hierarchy.

The classifier is stateless and safe for async contexts, inspecting only the exception object without maintaining internal state. When called, it constructs a lower-cased string combining the exception's type name and message content, then checks this against the keyword patterns defined in the rule table.

The Three-Step Classification Pattern

The workflow follows a predictable three-step pattern used throughout the graph layer:

1. Collect raw exceptions – Workflow modules like open_notebook/graphs/chat.py, source_chat.py, and ask.py catch exceptions from LangChain or Esperanto calls and forward them to classify_error.

2. Match against classification rules – The utility iterates through _CLASSIFICATION_RULES, checking if any keyword in the rule appears within the lower-cased exception string.

3. Return a predictable Open-Notebook error – The function returns a concrete subclass of OpenNotebookError (e.g., AuthenticationError, RateLimitError, NetworkError) together with a concise user-friendly message. Unrecognized errors are logged and wrapped as ExternalServiceError.

Step-by-Step Guide to Adding Custom Classifications

Step 1: Identify the New Failure Mode

Determine the distinctive phrases that appear in the raw exception text when the provider encounters the specific failure. The classifier converts the exception to a lower-cased string for matching, so you should identify unique keywords that appear in both the exception type name and message.

# Example: Extracting the search string

exception_str = str(exc).lower()

Step 2: Choose or Create an Exception Class

Select an appropriate subclass of OpenNotebookError from open_notebook/exceptions.py, or create a new one for provider-specific errors. Custom exceptions should inherit from the base OpenNotebookError class to ensure consistent handling by the API routers.

# In open_notebook/exceptions.py

class ProviderUnavailableError(OpenNotebookError):
    """Raised when the AI provider cannot allocate necessary resources."""
    pass

Step 3: Add a Classification Rule

Append a new tuple to _CLASSIFICATION_RULES in error_classifier.py. The tuple format is (keywords_list, ExceptionClass, user_message), where user_message can be None to forward the original exception text.

# At the bottom of _CLASSIFICATION_RULES in error_classifier.py

(
    ["gpu unavailable", "no gpu"],  # keywords to match

    ProviderUnavailableError,       # custom OpenNotebookError subclass

    "The provider cannot allocate GPU resources. Try again later.",  # friendly message

),

Step 4: Validate with Unit Tests

Create a test that raises a mock exception containing one of your keywords and asserts that classify_error returns the expected class and message. The tests/test_embedding.py file provides example patterns for verifying error classification behavior.

from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import ProviderUnavailableError

def test_gpu_unavailable_classification():
    class DummyExc(Exception):
        pass

    exc = DummyExc("GPU unavailable – please retry")
    exc_class, msg = classify_error(exc)

    assert exc_class is ProviderUnavailableError
    assert "GPU resources" in msg

Step 5: Deploy

No further changes are required to the graph modules or API routers. Because classify_error is imported throughout the graph layer (chat.py, ask.py, source_chat.py, transformation.py), the updated rule becomes instantly effective for all downstream endpoints including Chat, Ask, Source-Chat, and Search operations.

Code Implementation Examples

Calling the Classifier Directly

When implementing new provider integrations, wrap raw calls with the classifier to ensure consistent error handling:

from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import OpenNotebookError

try:
    # Some provider call that may raise a raw exception

    result = provider.generate(...)
except Exception as exc:
    exc_class, user_msg = classify_error(exc)
    # Re-raise a uniform Open-Notebook error

    raise exc_class(user_msg) from exc

Integration Points Across the Codebase

The classifier is imported and utilized across multiple critical graph modules:

• open_notebook/graphs/chat.py – Handles conversation flow errors
• open_notebook/graphs/source_chat.py – Manages source-specific chat failures
• open_notebook/graphs/ask.py – Processes question-answering errors
• open_notebook/graphs/transformation.py – Catches transformation pipeline failures

Because these modules rely on the centralized classify_error function, any new rule you add automatically extends error handling coverage to all existing endpoints without modifying individual call sites.

Summary

• Centralized architecture: All AI provider error classification flows through open_notebook/utils/error_classifier.py, ensuring consistent behavior across Chat, Ask, and Search workflows.
• Data-driven extensibility: Adding support for new failure modes requires only appending a tuple to _CLASSIFICATION_RULES, with no changes needed to the graph layer modules.
• Structured error hierarchy: Raw exceptions transform into specific OpenNotebookError subclasses (like AuthenticationError or RateLimitError) with user-friendly messages.
• Stateless design: The classifier safely operates in async contexts throughout the application, inspecting exception text without maintaining internal state.
• Comprehensive coverage: Unrecognized errors automatically fall back to ExternalServiceError, ensuring no raw provider exceptions leak to users unhandled.

Frequently Asked Questions

How does the classifier handle exceptions that don't match any known rules?

When classify_error encounters an exception that matches no keywords in _CLASSIFICATION_RULES, it logs the original exception for diagnostic purposes and returns ExternalServiceError along with the original exception message. This ensures users receive a generic but safe error notification while developers retain access to debugging information.

Can I use regular expressions instead of simple keyword matching in classification rules?

The current implementation in open_notebook/utils/error_classifier.py uses simple substring keyword matching against the lower-cased exception string. While the source code does not natively support regex patterns, you can implement complex matching logic by subclassing the exception handling in your specific graph module or by proposing a modification to the _CLASSIFICATION_RULES structure to support callable matchers.

Where should I define custom exception classes for new AI provider failures?

Define all custom exception classes in open_notebook/exceptions.py to maintain a clean separation of concerns. These classes should inherit from OpenNotebookError to ensure the API routers and frontend components recognize them as application-specific errors rather than unhandled system exceptions.

What is the performance impact of the error classification system?

The classifier introduces minimal overhead because it operates only when exceptions occur and performs simple string operations (lower-casing and substring searches) against a static rule table. Because the utility is stateless and does not perform I/O operations, it adds negligible latency to the error handling path even when processing high volumes of failed requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}