Error Classification for LLM Providers Using `classify_error()` in Open Notebook

The classify_error() utility transforms raw exceptions from LLM providers into standardized, user-friendly error types that prevent sensitive stack traces from reaching the UI while enabling precise HTTP error responses.

Open Notebook is an open-source project that abstracts interactions with various LLM providers through Esperanto and LangChain integrations. When underlying provider APIs throw exceptions, the classify_error() function in open_notebook/utils/error_classifier.py maps these low-level errors to a hierarchical exception system. This centralized approach ensures consistent error handling across AI provisioning, graph workflows, and REST API layers.

How classify_error() Works

Exception Hierarchy and Custom Types

The classification system relies on custom exception classes defined in open_notebook/exceptions.py. These include AuthenticationError, RateLimitError, ConfigurationError, NetworkError, and ExternalServiceError, all inheriting from a base OpenNotebookError. This hierarchy allows the application to catch specific error categories at different architectural layers while maintaining a clean separation between provider-specific details and domain logic.

The _CLASSIFICATION_RULES Mapping

At the core of open_notebook/utils/error_classifier.py (lines 19-69) is a static list _CLASSIFICATION_RULES that maps keyword groups to exception classes and optional friendly messages. The rules cover specific failure modes:

• Authentication failures (keywords: authentication, 401) → AuthenticationError
• Rate-limit hits (keywords: 429, quota exceeded) → RateLimitError
• Model misconfiguration (keyword: model not found) → ConfigurationError
• Network problems (keywords: timeout, connection refused) → NetworkError
• Context-length overruns (keywords: token limit, max_tokens) → ExternalServiceError
• Payload-size issues (keywords: 413, payload too large) → ExternalServiceError
• Provider outages (keywords: 500, service unavailable) → ExternalServiceError

Pattern Matching Algorithm

The matching algorithm converts both the exception's string representation and its class name to lowercase, concatenating them into a combined string. It scans each rule's keywords against this string, returning the first match. If a rule includes a custom message, that message takes precedence; otherwise, the original exception text passes through the _truncate helper to generate the user-facing output.

Fallback and Logging

When no patterns match, the function logs a warning at error_classifier.py:93 using logger.warning and defaults to ExternalServiceError with a truncated message. This ensures raw stack traces never leak to the UI, maintaining security and user experience standards.

Where classify_error() is Used in the Codebase

AI Model Provisioning

In open_notebook/ai/provision.py, provider-specific errors are caught and forwarded to classify_error() before raising an Open Notebook error. This isolates provider implementation details from the rest of the application, allowing the provisioning layer to switch between different LLM backends without changing error handling logic.

LangGraph Workflow Nodes

The graph nodes in open_notebook/graphs/* invoke LLM calls through LangGraph workflows. Any exception bubbling up from these asynchronous nodes is funneled through the classifier to maintain consistent error semantics across complex multi-step workflow execution.

API Service Layer

Services such as api/chat_service.py and api/models_service.py wrap provider calls with classify_error(), enabling FastAPI exception handlers to translate classified errors into clean HTTP 4xx/5xx responses with appropriate status codes.

Implementation Examples

The following patterns demonstrate how to integrate classify_error() into your service layer and API endpoints.

Direct use in a service:

from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import OpenNotebookError

def call_llm(provider, prompt):
    try:
        return provider.generate(prompt)
    except Exception as exc:
        exc_cls, message = classify_error(exc)
        raise exc_cls(message) from exc

Handling in a FastAPI endpoint:

from fastapi import APIRouter, HTTPException
from .services import call_llm

router = APIRouter()

@router.post("/ask")
async def ask_endpoint(payload: dict):
    try:
        answer = await call_llm(payload["provider"], payload["prompt"])
        return {"answer": answer}
    except OpenNotebookError as e:
        raise HTTPException(status_code=400, detail=str(e))

Unit-test expectations:

import pytest
from open_notebook.utils.error_classifier import classify_error
from open_notebook.exceptions import RateLimitError

def test_rate_limit_classification():
    raw_exc = Exception("429 Too Many Requests")
    exc_cls, msg = classify_error(raw_exc)
    assert exc_cls is RateLimitError
    assert "Rate limit exceeded" in msg

Summary

• classify_error() centralizes error mapping from raw LLM provider exceptions to typed Open Notebook errors.
• _CLASSIFICATION_RULES in open_notebook/utils/error_classifier.py defines keyword-based mappings for authentication failures, rate limits, network issues, and configuration errors.
• The function returns a tuple of (exception_class, user_message) that the API layer converts into appropriate HTTP responses.
• A fallback mechanism ensures that unmatched exceptions become ExternalServiceError with truncated messages, preventing stack trace leakage.

Frequently Asked Questions

What types of errors can classify_error() detect?

The function detects authentication failures (401), rate limits (429), model configuration errors, network timeouts, token limit violations, and provider outages (500). Each maps to specific exception classes in open_notebook/exceptions.py based on keyword matching against the raw exception text.

How does classify_error() handle unknown exceptions?

When no classification rules match, the function logs a warning to error_classifier.py:93 and returns a generic ExternalServiceError with a truncated version of the original message. This ensures the application degrades gracefully without exposing internal stack traces to end users.

Where should I add new classification rules for a new provider?

Add new keyword patterns to the _CLASSIFICATION_RULES list in open_notebook/utils/error_classifier.py (lines 19-69). Insert the new rule before the generic fallbacks, specifying the exception class from open_notebook/exceptions.py and an optional user-friendly message string.

Why does Open Notebook truncate error messages?

The _truncate helper limits message length to prevent oversized error payloads from reaching the frontend and to strip potentially sensitive internal details. This keeps HTTP responses lightweight and ensures that only actionable, user-friendly information appears in the UI.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}