Open Notebook Testing Strategy: How to Run the Pytest Suite

Open Notebook implements a layered testing strategy using pytest to validate unit logic, graph workflows, and utility functions, with all tests executed via uv run pytest from the tests/ directory.

Open Notebook, an open-source AI notebook application built on LangGraph, relies on a comprehensive testing strategy to ensure reliability across its domain models, API contracts, and workflow orchestration. The project uses pytest as its test runner, configured specifically for async Python execution to support the asynchronous database and API layers. Understanding how to run the pytest suite and navigate the test structure is essential for contributors working with the lfnovo/open-notebook codebase.

Understanding the Layered Testing Strategy

The testing strategy documented in open_notebook/CLAUDE.md organizes validation into four distinct layers. Each layer targets specific components of the application, from individual domain models to end-to-end workflow execution.

Unit Tests

Unit tests cover individual domain models and service functions in isolation. These tests validate notebook validation logic, archiving features, and API model contracts without external dependencies.

Key files include:

• tests/test_domain.py
• tests/test_models_api.py

Graph Tests

Graph tests provide end-to-end validation of LangGraph workflow execution, ensuring state-machine orchestration behaves correctly across complex multi-step AI processes.

Key file:

• tests/test_graphs.py

Utility Tests

Utility tests verify helper modules responsible for text processing, embedding generation, and data chunking. These ensure core preprocessing functions remain stable.

Key files include:

• tests/test_utils.py
• tests/test_chunking.py
• tests/test_embedding.py

Integration Tests

Integration tests validate interactions between system layers and external services. These are organized under tests/integration/ and verify that components work together correctly.

Key Testing Infrastructure Files

The testing configuration and documentation reside in specific locations:

Documentation:

• open_notebook/CLAUDE.md – Contains the Testing Strategy section outlining the layered approach
• docs/7-DEVELOPMENT/testing.md – Provides detailed execution guides and fixture patterns

Configuration:

• pyproject.toml and uv.lock – Define the Python environment and test dependencies
• tests/ – Root directory containing all test modules organized by category

Running the Pytest Suite

The official testing guide specifies using uv for environment management and pytest for execution. All commands assume execution from the repository root.

Execute All Tests

Run the complete test suite using uv to ensure dependency isolation:

uv run pytest

Run Specific Test Categories

Target individual test files or directories to narrow scope:

# Run a specific test file

uv run pytest tests/test_notebooks.py

# Run only unit tests

uv run pytest tests/unit/

# Run only graph tests

uv run pytest tests/test_graphs.py

Execute Single Test Functions

Isolate specific test cases using Python's standard pytest syntax:

uv run pytest tests/test_notebooks.py::test_create_notebook

Generate Coverage Reports

Measure code coverage for the open_notebook package:

uv run pytest --cov=open_notebook

Additional Useful Flags

Enhance test output with these common options:

• -v – Verbose output showing each test name
• -s – Display print statements and stdout during execution

Example with verbose output:

uv run pytest -v tests/test_graphs.py

Writing Async Tests

All tests utilize async support via @pytest.mark.asyncio because the core API and database layers are asynchronous. Tests must be decorated to enable async execution.

Example test pattern from the documentation:

import pytest
from open_notebook.domain.notebook import Notebook
from open_notebook.errors import InvalidInputError

@pytest.mark.asyncio
async def test_notebook_validation():
    """Invalid name should raise an error."""
    with pytest.raises(InvalidInputError):
        Notebook(name="", description="demo")

The testing guide recommends using fixtures for shared setup and teardown, and asserting both success paths and error conditions to maintain robust coverage.

Summary

• Open Notebook uses a layered testing strategy with distinct categories: unit, graph, utility, and integration tests.
• The pytest suite is executed via uv run pytest to ensure reproducible environments defined in pyproject.toml.
• All tests are async and require the @pytest.mark.asyncio decorator to match the application's async runtime.
• Test files are organized under tests/ with specific subdirectories for unit and integration tests.
• Coverage reports are generated using the --cov=open_notebook flag.
• Strategy documentation lives in open_notebook/CLAUDE.md while execution details are in docs/7-DEVELOPMENT/testing.md.

Frequently Asked Questions

What testing frameworks does Open Notebook use?

The project uses pytest as the primary test runner, configured for async Python support via @pytest.mark.asyncio. Environment management is handled by uv, which reads dependencies from pyproject.toml and uv.lock to ensure consistent test execution across development machines.

How do I run only the graph workflow tests?

Execute uv run pytest tests/test_graphs.py to run only the LangGraph workflow validations. For verbose output showing each graph test name, add the -v flag: uv run pytest -v tests/test_graphs.py.

Why are all tests async?

The core API and database layers in Open Notebook are built on asynchronous Python patterns. Therefore, all tests use @pytest.mark.asyncio to properly handle async/await syntax in test functions, ensuring accurate validation of the actual runtime behavior.

Where is the testing strategy documented?

The high-level Testing Strategy is documented in open_notebook/CLAUDE.md under the Testing Strategy section. Detailed execution commands, fixture patterns, and best practices are outlined in docs/7-DEVELOPMENT/testing.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}