How to Use the Open Notebook REST API for Programmatic Notebook and Source Management

Open Notebook exposes a FastAPI-based HTTP interface that enables external programs to create, read, update, and delete notebooks and sources asynchronously, with dedicated endpoints for file uploads, graph-based linking, and cascade deletion.

The Open Notebook project provides a privacy-first research assistant with a comprehensive REST API for programmatic notebook and source management. This FastAPI-driven interface, available in the lfnovo/open-notebook repository, enables external applications to automate research workflows, manage knowledge graphs, and handle long-running document processing jobs without blocking client requests. All endpoints are asynchronous and delegate persistence to SurrealDB through a centralized repository pattern.

API Architecture and Core Components

The REST API follows a layered architecture that separates HTTP handling from domain logic and data persistence.

HTTP Layer and Routing

The FastAPI application organizes endpoints into dedicated router modules. In api/routers/notebooks.py, the application implements CRUD operations for notebooks, optional filtering and sorting for list views, and graph-based source linking. The api/routers/sources.py module handles multipart file uploads, async processing coordination, status polling, and file downloads.

Both routers delegate business logic to domain models and use Pydantic schemas defined in api/models.py for request validation and response serialization.

Domain Models and Persistence

Domain logic resides in open_notebook/domain/notebook.py and open_notebook/domain/source.py. These modules implement business rules for persistence, reference handling, and cascade operations. The open_notebook/database/repository.py file provides a thin wrapper around SurrealDB through functions like repo_query and ensure_record_id, which build safe, parameterized queries to prevent injection attacks.

Asynchronous Processing Pipeline

Long-running operations such as text extraction, transformation, and vector embedding are handled through the api/command_service.py module. When async_processing is enabled, the API calls CommandService.submit_command_job to queue a command in the Surreal Commands queue rather than blocking the request. For synchronous processing, the API uses execute_command_sync within a thread pool.

Managing Notebooks via the REST API

Notebooks serve as the primary organizational units, maintaining references to sources through graph edges rather than direct ownership.

Creating and Listing Notebooks

To create a notebook, POST to the /notebooks endpoint with a JSON payload containing name and optional description. The endpoint returns a NotebookResponse object with the generated id.

Listing supports query parameters for filtering and sorting. The endpoint validates the order_by parameter against an allow-list in api/routers/notebooks.py before interpolating it into SurrealQL queries, preventing injection attacks. List responses include computed fields like source_count and note_count derived from graph edge counts.

Linking Sources to Notebooks

Sources can belong to zero or many notebooks. The dedicated endpoint POST /notebooks/{notebook_id}/sources/{source_id} creates a graph edge linking an existing source to a notebook without duplicating the source record. This maintains the "resource-first" design where sources exist independently and are referenced by multiple notebooks.

Cascade Deletion

The DELETE endpoint /notebooks/{notebook_id} accepts a delete_exclusive_sources boolean parameter. When true, the cascade logic in open_notebook/domain/notebook.py removes sources that are exclusively referenced by the deleted notebook while unlinking shared sources. The response includes counts of deleted notes, exclusive sources removed, and sources merely unlinked.

Managing Sources via the REST API

Sources represent research materials and support multiple content types including text, URLs, and uploaded files.

Text Sources and Synchronous Processing

For text-only sources, POST to /sources with "type": "text" and the content in the payload. Omitting async_processing or setting it to false triggers synchronous processing via execute_command_sync. The response immediately includes the extracted full_text, embedding status, and a null command_id.

File Uploads and Asynchronous Processing

For file uploads, use multipart/form-data with type=upload and include the file field. Setting async_processing=true queues the job via CommandService.submit_command_job and returns immediately with:

• command_id: The background job identifier
• status: "new" (job queued)
• processing_info: Object indicating async queue status

The file is stored under UPLOADS_FOLDER with a unique filename generated by generate_unique_filename to prevent collisions.

Polling for Processing Status

Async sources can be polled via GET /sources/{source_id}/status or GET /sources/{source_id}. The status endpoint returns the current state (queued, running, completed, or failed) and a human-readable message. Upon completion, the full source record includes the extracted text and embedded: true.

Downloading Original Files

The endpoint GET /sources/{source_id}/download returns the original uploaded file as a FileResponse. The implementation in api/routers/sources.py uses _resolve_source_file to validate paths through generate_unique_filename, ensuring resolved paths remain within UPLOADS_FOLDER to prevent directory traversal attacks.

Security and Data Integrity

The API implements several safety measures at the repository and router layers. SurrealQL injection protection validates sort parameters against allow-lists before query construction. File path sanitization ensures uploads and downloads are restricted to the designated upload directory. The ensure_record_id helper in open_notebook/database/repository.py validates record identifiers before database operations.

Practical API Examples

The following curl commands demonstrate common operations against a local server running on port 5055.

Create a notebook:

curl -X POST http://localhost:5055/notebooks \
  -H "Content-Type: application/json" \
  -d '{
        "name": "Research on AI Safety",
        "description": "Collected papers and notes"
      }'

List notebooks with filtering:

curl http://localhost:5055/notebooks?archived=false\&order_by=name\ asc

Link an existing source to a notebook:

curl -X POST http://localhost:5055/notebooks/nb456/sources/src123

Create a synchronous text source:

curl -X POST http://localhost:5055/sources \
  -H "Content-Type: application/json" \
  -d '{
        "type": "text",
        "content": "Open‑Notebook is a privacy‑first research assistant.",
        "title": "About Open‑Notebook",
        "notebooks": ["nb456"],
        "embed": true
      }'

Upload a file with async processing:

curl -X POST http://localhost:5055/sources \
  -F type=upload \
  -F title="Sample PDF" \
  -F notebooks='["nb456"]' \
  -F async_processing=true \
  -F embed=true \
  -F file=@/path/to/document.pdf

Check processing status:

curl http://localhost:5055/sources/<source_id>/status

Download original file:

curl -OJ http://localhost:5055/sources/<source_id>/download

Delete notebook with cascade:

curl -X DELETE "http://localhost:5055/notebooks/nb456?delete_exclusive_sources=true"

Summary

• Open Notebook provides a FastAPI-based REST API for full CRUD operations on notebooks and sources, accessible via api/routers/notebooks.py and api/routers/sources.py.
• The architecture separates HTTP handling from domain logic (open_notebook/domain/) and SurrealDB persistence (open_notebook/database/repository.py).
• Async processing via CommandService.submit_command_job prevents blocking during text extraction and embedding, with status polling available through dedicated endpoints.
• Graph-based relationships allow sources to link to multiple notebooks without duplication, cascade deletion respects exclusive vs. shared references.
• Security measures include SurrealQL injection protection, path validation in generate_unique_filename, and directory traversal prevention in _resolve_source_file.

Frequently Asked Questions

How does the API handle long-running document processing?

When async_processing is set to true in api/routers/sources.py, the endpoint writes a minimal source record and immediately queues a command via CommandService.submit_command_job. The client receives a command_id and initial status of "new". The background job handles text extraction, transformations, and vector embedding. Clients poll /sources/{source_id}/status to monitor progress through states (queued, running, completed, failed) without blocking the original request.

Can a single source belong to multiple notebooks simultaneously?

Yes. The data model in open_notebook/domain/source.py treats sources as independent resources. The POST /notebooks/{notebook_id}/sources/{source_id} endpoint creates a graph edge referencing the source without duplicating the record. This allows sources to be linked to zero or many notebooks. When deleting a notebook, the cascade logic in open_notebook/domain/notebook.py distinguishes between exclusive sources (deleted) and shared sources (unlinked).

What security measures protect against injection attacks and path traversal?

The API implements SurrealQL injection protection by validating order_by parameters against strict allow-lists before interpolation in api/routers/notebooks.py. For file operations, generate_unique_filename in the upload handler and _resolve_source_file in the download endpoint both resolve real paths and verify they remain within UPLOADS_FOLDER. The ensure_record_id helper in open_notebook/database/repository.py sanitizes record identifiers before database queries.

How do I retrieve the full text and embedding status of a processed source?

Query GET /sources/{source_id} to retrieve the complete source record. For async sources, once the command completes, the response includes full_text containing extracted content and embedded: true indicating successful vectorization. The /sources/{source_id}/status endpoint provides a lightweight alternative that returns only the processing state and message without the full document payload.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}