Open Notebook Database Schema for Notebooks, Sources, and Notes: A Complete Guide

Open Notebook stores notebooks, sources, and notes as SurrealDB graph record types, linking them via reference, artifact, and refers_to edges defined in the Pydantic models located in open_notebook/domain.

The lfnovo/open-notebook application uses SurrealDB—a graph-oriented database—to persist its core content hierarchy. The database schema for notebooks, sources, and notes is encoded in Python Pydantic models that are materialized on first save and connected through explicit graph edges. These record types and their relationships power every CRUD operation exposed by the FastAPI backend.

Core Record Types in the SurrealDB Schema

Open Notebook defines three primary tables: notebook, source, and note. Each table acts as a record type with timestamp fields and optional metadata.

Notebook Records

In open_notebook/domain/notebook.py, the Notebook model declares the following schema:

• id: str — the SurrealDB record ID, populated after save().
• name: str — the display name of the collection.
• description: str — a user-provided summary.
• archived: bool — defaults to False; soft-delete flag.
• created: datetime — auto-generated creation timestamp.
• updated: datetime — auto-generated last-modified timestamp.

A notebook represents a user-owned collection that aggregates sources and notes through outgoing edges.

Source Records

In open_notebook/domain/source.py, the Source model stores raw imported content:

• id: str — the record identifier.
• asset: Asset (optional) — contains either a file_path (str) or url (str).
• title: str (optional) — extracted or user-provided title.
• topics: list[str] — defaults to []; labels for categorization.
• full_text: str (optional) — the extracted body content.
• command: RecordID (optional) — reference to an async processing job.
• created: datetime — creation timestamp.
• updated: datetime — modification timestamp.

Sources serve as the canonical containers for imported files, URLs, and other external materials.

Note Records

In open_notebook/domain/note.py, the Note model captures human- and AI-authored text:

• id: str — the record identifier.
• title: str (optional) — note headline.
• note_type: Literal["human", "ai"] (optional) — provenance indicator.
• content: str (optional) — the textual body.
• created: datetime — creation timestamp.
• updated: datetime — modification timestamp.

Notes are lightweight text artifacts that attach to one or more notebooks.

Graph Relationships and Edge Types

SurrealDB treats relationships as first-class graph edges. Open Notebook uses three primary edge types to link record types.

Reference Edges (notebook → source)

The reference edge creates a many-to-many link from a notebook to its sources. A single source may be referenced by multiple notebooks.

In open_notebook/domain/source.py, the Source.add_to_notebook(notebook_id) method invokes ObjectModel.relate("reference", notebook_id) to persist this edge. To traverse the relationship in reverse, Notebook.get_sources() in open_notebook/domain/notebook.py (lines 29–61) executes a SurrealQL query that returns List[Source].

Artifact Edges (notebook → note)

The artifact edge creates a many-to-many link from a notebook to its notes.

In open_notebook/domain/note.py, Note.add_to_notebook(notebook_id) calls ObjectModel.relate("artifact", notebook_id). The notebook side resolves these links through Notebook.get_notes(), implemented alongside get_sources() in the notebook model.

Refers_to Edges (chat_session → notebook or source)

The refers_to edge optionally scopes a chat session to a specific notebook or source. This enables contextual conversation threads without embedding chat data directly inside the content records.

Source-specific Derived Edges

Sources also participate in two self-referential or derivative edge types:

• source_embedding → source: stores vector embeddings for text chunks.
• source_insight → source: stores AI-generated insights derived from the content.

These edges are managed through the source model and its helpers.

Schema Implementation in the Repository Layer

Low-level SurrealDB operations live in open_notebook/database/repository.py. This module provides repo_query, repo_create, repo_update, and repo_relate, which the domain models call to materialize the schema.

All three domain models inherit persistence behavior from ObjectModel:

• Creating a record — Notebook.save(), Source.save(), and Note.save() delegate to ObjectModel.save, which issues a SurrealDB CREATE or UPDATE.
• Relating records — ObjectModel.relate(edge_name, target_id) constructs the graph link.
• Fetching relations — Notebook.get_sources() and Notebook.get_notes() build SurrealQL SELECT statements that traverse outgoing edges.

Practical Examples: Creating and Linking Records

The following examples demonstrate how to instantiate the schema in practice.

Create a Notebook and a Source

# Example: create a notebook, a source, and link them

from open_notebook.domain.notebook import Notebook
from open_notebook.domain.source import Source, Asset

# 1️⃣ Create a notebook

nb = Notebook(name="Research on AI Ethics", description="Collected articles and notes")
await nb.save()                     # persisted → nb.id populated

# 2️⃣ Create a source (e.g., from a URL)

src = Source(
    title="AI Ethics Paper",
    full_text="Full text of the paper …",
    asset=Asset(url="https://example.com/ai-ethics.pdf")
)
await src.save()                    # persisted → src.id populated

# 3️⃣ Link source to notebook

await src.add_to_notebook(nb.id)    # creates a `reference` edge

# 4️⃣ Retrieve sources for the notebook

sources = await nb.get_sources()    # returns List[Source]

print([s.title for s in sources])

Create a Note and Attach It to a Notebook

# Example: create a note and attach it to a notebook

from open_notebook.domain.note import Note

note = Note(title="Key Takeaways", note_type="human", content="…")
await note.save()
await note.add_to_notebook(nb.id)   # creates an `artifact` edge

# Fetch notes belonging to the notebook

notes = await nb.get_notes()
print([n.title for n in notes])

Delete a Notebook with Cascade Preview

The notebook model supports scoped deletion that previews exclusive versus shared sources.

# Example: remove a notebook while optionally deleting exclusive sources

preview = await nb.get_delete_preview()
print(preview)   # {'note_count': 3, 'exclusive_source_count': 2, 'shared_source_count': 5}

# Delete notebook, also removing sources that are exclusive to it

result = await nb.delete(delete_exclusive_sources=True)
print(result)    # {'deleted_notes': 3, 'deleted_sources': 2, 'unlinked_sources': 5}

Key Files That Define the Schema

The following files collectively encode the database schema and the CRUD pathways:

• open_notebook/domain/notebook.py — Pydantic model for notebook, relationship helpers, and deletion logic.
• open_notebook/domain/source.py — Model for source, asset handling, embedding, insight creation, and notebook linking.
• open_notebook/domain/note.py — Model for note; provides saving, embedding submission, and notebook linking.
• open_notebook/database/repository.py — Low-level SurrealDB helpers (repo_query, repo_create, repo_update, repo_relate).
• api/routers/notebooks.py — FastAPI endpoints for notebook CRUD.
• api/routers/sources.py — FastAPI endpoints for source operations.
• api/routers/notes.py — FastAPI endpoints for note operations.

Summary

• Open Notebook uses SurrealDB graph record types for notebook, source, and note entities.
• The schema fields are defined in Pydantic models under open_notebook/domain and materialized through save() operations.
• reference edges link notebooks to sources in a many-to-many relationship.
• artifact edges link notebooks to notes in a many-to-many relationship.
• refers_to edges optionally bind chat sessions to notebooks or sources.
• Utility methods such as get_sources(), get_notes(), and get_delete_preview() encapsulate SurrealQL traversal and scoped deletion logic.

Frequently Asked Questions

What database does Open Notebook use?

Open Notebook uses SurrealDB, a graph-oriented database. Each table acts as a record type, and relationships are stored as first-class graph edges rather than foreign keys.

How are notebooks linked to sources and notes in the schema?

Notebooks connect to sources through reference edges and to notes through artifact edges. Both relationships are many-to-many, managed via ObjectModel.relate() in the domain layer and queried through Notebook.get_sources() and Notebook.get_notes().

Can a source belong to multiple notebooks?

Yes. Because the notebook-to-source link is a reference edge, a single source record can be related to many notebooks. Deleting a notebook can optionally remove exclusive sources while leaving shared sources intact, as shown in Notebook.delete(delete_exclusive_sources=True).

Where is the schema defined in the codebase?

The schema is defined in Python Pydantic models inside open_notebook/domain/notebook.py, open_notebook/domain/source.py, and open_notebook/domain/note.py. The actual SurrealDB persistence is handled by open_notebook/database/repository.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}