Testing Strategies for API Endpoints and LangGraph Graphs in Open-Notebook

Open-Notebook employs FastAPI TestClient for REST endpoint validation and unit-level mocking for LangGraph workflows, ensuring deterministic, isolated tests across async database operations and AI provider integrations.

The lfnovo/open-notebook repository implements a comprehensive testing framework that validates both its FastAPI-based REST endpoints and LangGraph-based AI workflows. By combining HTTP-level integration tests with isolated unit tests for graph nodes, the codebase ensures reliable handling of asynchronous database operations and external AI provider integrations. This article examines the specific testing strategies used to verify request contracts, business logic constraints, and complex workflow execution paths.

API Endpoint Testing Strategies

FastAPI TestClient for HTTP Contract Validation

The test suite utilizes FastAPI's TestClient to exercise full HTTP request lifecycles without requiring a live server. Following the Arrange-Act-Assert pattern, tests in tests/test_models_api.py instantiate the client and send real HTTP requests to validate routing, Pydantic validation, and CORS configuration.

from fastapi.testclient import TestClient
from open_notebook.api.main import app

client = TestClient(app)

def test_create_model():
    response = client.post("/api/models", json={
        "name": "gpt-4-test",
        "provider": "openai",
        "type": "text"
    })
    assert response.status_code == 200

Isolating Business Logic with unittest.mock

Tests avoid SurrealDB side effects by mocking the repository layer and model persistence methods. The suite patches open_notebook.database.repository.repo_query and api.routers.models.Model.save to verify business rules like duplicate detection and case-insensitivity without database I/O.

from unittest.mock import patch

@patch("open_notebook.database.repository.repo_query")
@patch("api.routers.models.Model.save")
def test_model_duplicate_validation(mock_save, mock_repo):
    mock_repo.return_value = [{"id": "existing-model"}]
    response = client.post("/api/models", json={"name": "duplicate-model"})
    assert response.status_code == 400

Environment Variable and Provider Configuration Testing

Endpoints exposing provider availability depend on environment variables. Tests patch os.environ.get and AIFactory.get_available_providers to simulate configurations without requiring actual API keys, validating both generic and mode-specific variable handling.

@patch("api.routers.models.os.environ.get")
@patch("api.routers.models.AIFactory.get_available_providers")
def test_provider_availability(mock_providers, mock_env):
    mock_env.return_value = "test-api-key"
    mock_providers.return_value = ["openai", "anthropic"]
    response = client.get("/api/models/providers")
    assert "openai" in response.json()

Async Endpoint Testing with pytest-asyncio

For endpoints handling asynchronous operations like source ingestion, tests use @pytest.mark.asyncio and AsyncMock to emulate async database calls. This pattern appears in tests/test_models_api.py for validating non-blocking I/O patterns and async model persistence.

import pytest
from unittest.mock import AsyncMock, patch

@pytest.mark.asyncio
async def test_async_model_save():
    with patch.object(Model, "save", new_callable=AsyncMock) as mock_save:
        mock_save.return_value = {"id": "async-model"}
        response = await client.post("/api/models", json={"name": "async-test"})
        assert response.status_code == 200

Error Handling and Edge Case Coverage

The suite validates HTTP status codes and error messages for failure scenarios. In tests/test_crud_404.py, tests confirm that missing resources return 404 status codes, while test_models_api.py verifies 400 responses for validation failures and duplicate entries.

def test_missing_resource_returns_404():
    response = client.get("/api/models/non-existent-id")
    assert response.status_code == 404

LangGraph Workflow Testing Strategies

Validating Tool Definitions and Metadata

Custom LangGraph tools undergo strict output validation. In tests/test_graphs.py, the get_current_timestamp tool is exercised to verify its 14-character format and proper LangGraph metadata (name, description).

from open_notebook.graphs.tools import get_current_timestamp

def test_timestamp_tool_format():
    timestamp = get_current_timestamp.func()
    assert len(timestamp) == 14
    assert timestamp.isdigit()

State Object Structure Verification

Typed state dictionaries like PatternChainState and TransformationState are instantiated to guarantee field names and defaults match the graph's expected schema, ensuring correct data passing between nodes.

from open_notebook.graphs.transformation import PatternChainState

def test_state_object_mapping():
    state = PatternChainState(prompt="test prompt", content="test content")
    assert state["prompt"] == "test prompt"
    assert "content" in state

Graph Compilation and Method Availability

Each compiled graph is tested to confirm successful DAG construction and availability of invocation methods. Tests verify that graph and transformation_graph objects expose both invoke and ainvoke methods as implemented in the LangGraph compilation.

from open_notebook.graphs.transformation import transformation_graph

def test_graph_compilation():
    assert hasattr(transformation_graph, "invoke")
    assert hasattr(transformation_graph, "ainvoke")

Async Node Execution and Error Handling

The run_transformation async function is tested with invalid inputs to verify assertion failures and error messaging. Tests confirm that missing content triggers a clear AssertionError with the message "No content to transform".

import pytest
from open_notebook.graphs.transformation import run_transformation

@pytest.mark.asyncio
async def test_transformation_empty_content():
    state = {"content": ""}
    config = {}
    with pytest.raises(AssertionError, match="No content to transform"):
        await run_transformation(state, config)

Source Title Preservation Logic

The save_source node logic is isolated to verify title handling across scenarios: custom titles, placeholder "Processing..." strings, and empty values. Tests patch Source.get to ensure the node respects user-defined titles while replacing placeholders appropriately.

from unittest.mock import patch, MagicMock

def test_save_source_title_preservation():
    mock_source = MagicMock()
    mock_source.title = "Processing..."
    
    with patch("open_notebook.graphs.source.Source.get", return_value=mock_source):
        result = save_source_node(state={"title": "My Custom Research Title"})
        assert mock_source.title == "My Custom Research Title"

Summary

• FastAPI TestClient exercises full HTTP request lifecycles in tests/test_models_api.py, validating routing, validation, and CORS without external servers.
• pytest fixtures provide shared client setup across the test suite, adhering to DRY principles.
• unittest.mock and AsyncMock isolate database operations and AI provider calls, ensuring deterministic tests for async endpoints and business logic without SurrealDB I/O.
• Environment variable patching simulates different provider configurations in AIFactory.get_available_providers without requiring real API credentials.
• LangGraph validation covers tool definitions, state object schemas (PatternChainState, TransformationState), and graph compilation in tests/test_graphs.py.
• Async node testing uses pytest.mark.asyncio to verify error handling and data flow in workflow graphs like transformation_graph.
• Edge case coverage includes duplicate detection, 404 handling, title preservation, empty content validation, and placeholder replacement logic.

Frequently Asked Questions

How does Open-Notebook test FastAPI endpoints without a database?

The test suite uses unittest.mock to patch repo_query and Model.save methods, simulating SurrealDB responses without connecting to a real database. This approach allows tests in tests/test_models_api.py to verify HTTP contracts and business logic in isolation using the Arrange-Act-Assert pattern.

What framework handles async testing for LangGraph workflows?

Tests utilize pytest.mark.asyncio decorators combined with AsyncMock to handle async graph nodes. The run_transformation function and other async workflows are tested by mocking coroutines and asserting on state transitions without executing real I/O operations against external AI services.

How are environment-specific configurations tested for AI providers?

Tests patch os.environ.get and AIFactory.get_available_providers to simulate different API key configurations and provider availability. This ensures endpoints like /api/models correctly expose provider lists based on environment variables without requiring actual provider credentials.

Where are the LangGraph tool definitions validated?

Custom tool validation occurs in tests/test_graphs.py, where functions like get_current_timestamp are exercised to verify 14-character output formats, metadata compliance, and LangGraph compatibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}