How Open Notebook's Error Classifier Maps LLM Exceptions to User-Friendly Messages

Open Notebook's classify_error utility in open_notebook/utils/error_classifier.py intercepts low-level LLM provider exceptions, matches them against keyword-based classification rules, and returns standardized user-friendly messages while preserving technical details for logging.

Open Notebook integrates with multiple AI providers through the Esperanto library, but raw exceptions from these services often contain confusing technical details or sensitive information. The error classification system transforms these low-level failures into clear, actionable messages that help users understand what went wrong without exposing internal implementation details.

The Challenge of Provider-Specific Exceptions

When working with external LLM providers, applications encounter diverse failure modes ranging from authentication errors to rate limits and network timeouts. Exposing raw exception text directly to end users creates a poor experience and risks leaking sensitive API keys or internal stack traces. Open Notebook solves this through a centralized error classification utility that standardizes error handling across the entire application.

Core Architecture of the Error Classifier

Located in open_notebook/utils/error_classifier.py, the classification system revolves around the classify_error function. This utility normalizes exception inputs and maps them against the _CLASSIFICATION_RULES table to determine the appropriate user-facing message and exception type.

Normalizing Exception Input

The classification process begins by combining the exception's class name with its string representation, then converting the result to lowercase. This normalization ensures consistent matching regardless of how the provider formats their error messages, creating a unified text string that the rule engine can evaluate reliably.

Pattern Matching Against Classification Rules

The _CLASSIFICATION_RULES table contains entries that specify:

• Keyword substrings to search for in the normalized error text
• Target exception classes (all inheriting from OpenNotebookError defined in open_notebook/exceptions.py)
• Optional human-readable messages for specific error categories

The rules cover common failure modes including authentication failures, rate limiting, configuration errors, network timeouts, context-length limits, and payload-size violations.

Returning User-Friendly Messages

The function returns a tuple containing (exception_class, user_message). When a rule provides a custom message, that text is returned directly; otherwise, the original exception is truncated to 200 characters using the _truncate helper to prevent verbose internal details from reaching the user interface. Unclassified errors trigger a warning log and fall back to the generic ExternalServiceError class.

Integration Across the Application Stack

The error classifier operates at multiple layers of the Open Notebook architecture, ensuring consistent error handling from graph nodes to API endpoints.

Graph Node Implementation

All higher-level LangGraph nodes wrap their provider calls in try/except blocks and delegate exceptions to the classifier. Files such as open_notebook/graphs/chat.py, open_notebook/graphs/source_chat.py, ask.py, and transformation.py follow this pattern to ensure that LLM failures are transformed before propagating upward.

API Layer Exception Handling

API routers including api/routers/source_chat.py and api/routers/search.py catch exceptions from graph nodes, invoke classify_error, and raise the returned exception class. FastAPI's custom exception handlers defined in api/main.py then map these OpenNotebookError instances to appropriate HTTP status codes while preserving CORS headers for cross-origin requests.

Practical Implementation Examples

Use the classifier directly when handling provider exceptions:

from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import AuthenticationError, RateLimitError

# Simulate a low-level provider exception

class ProviderError(RuntimeError):
    pass

try:
    # Imagine a call to an LLM provider that raises an exception

    raise ProviderError("401 Unauthorized – invalid_api_key")
except BaseException as exc:
    err_class, user_msg = classify_error(exc)
    # err_class will be AuthenticationError, user_msg the friendly hint

    print(err_class)   # <class 'open_notebook.exceptions.AuthenticationError'>

    print(user_msg)    # Authentication failed. Please check your API key in Settings -> Credentials.

In a FastAPI endpoint, the pattern ensures proper HTTP responses:

from fastapi import APIRouter, HTTPException
from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import OpenNotebookError

router = APIRouter()

@router.get("/example")
async def example_endpoint():
    try:
        # Call into a LangGraph node that talks to an LLM

        result = await some_graph_node.invoke(...)
        return {"result": result}
    except Exception as exc:
        err_cls, user_msg = classify_error(exc)
        # Re-raise the mapped OpenNotebookError for FastAPI to handle

        raise err_cls(user_msg) from exc

Summary

• The classify_error function in open_notebook/utils/error_classifier.py provides centralized exception mapping for all LLM provider interactions.
• Input normalization combines exception class names and messages to create consistent matching criteria.
• The _CLASSIFICATION_RULES table maps keywords to specific exception classes like AuthenticationError and RateLimitError with human-readable messages.
• Uncaught errors fall back to ExternalServiceError with truncated messages to prevent information leakage.
• Graph nodes in chat.py, source_chat.py, and other files use the classifier to sanitize errors before they reach the API layer.
• FastAPI handlers in api/main.py convert classified exceptions into appropriate HTTP responses with proper CORS headers.

Frequently Asked Questions

What happens when an error doesn't match any classification rules?

When the classifier encounters an exception that doesn't match any keywords in _CLASSIFICATION_RULES, it logs a warning message and returns the generic ExternalServiceError class paired with the original error message truncated to 200 characters. This ensures users receive a generic but safe error notification while developers retain debugging information through logs.

How does the error classifier prevent sensitive information leakage?

The system employs two protective mechanisms: automatic truncation of unknown exceptions to 200 characters via the _truncate function, and the use of predefined human-readable messages for known error categories. This prevents raw stack traces, API keys, or internal implementation details from appearing in user-facing interfaces while preserving technical data in server logs.

Which Open Notebook components use the error classifier?

The classifier is utilized across the entire application stack. Graph nodes including open_notebook/graphs/chat.py, source_chat.py, ask.py, and transformation.py wrap LLM calls with the classifier. API routers such as api/routers/source_chat.py and api/routers/search.py intercept exceptions before they reach the HTTP response layer, ensuring consistent error formatting across the application.

What exception types are available in the Open Notebook hierarchy?

All classified exceptions inherit from the base OpenNotebookError class defined in open_notebook/exceptions.py. Specific subclasses include AuthenticationError for API key failures, RateLimitError for throttling issues, and ExternalServiceError as the generic fallback. Each subclass can carry custom user-facing messages that explain the specific failure mode in clear, actionable language.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}