SurrealDB Graph Relationships in Open Notebook: Notebooks, Sources, and Notes

Open Notebook leverages SurrealDB graph edges to link notebooks to sources via reference relationships and to notes via artifact relationships, enabling bidirectional traversal, fine-grained cascade control, and semantic search integration.

Open Notebook stores all user data in SurrealDB, a graph-oriented database that treats relationships as first-class entities rather than foreign keys. The architecture centers on three primary record types—Notebooks, Sources, and Notes—connected through explicit edges that form the backbone of the application's knowledge graph and research workflow.

Core Graph Schema: Tables and Edge Types

Open Notebook defines three main tables that serve as nodes in the knowledge graph. These entities are linked by directional edges that SurrealDB stores as distinct records, allowing complex traversal queries without JOIN operations.

Primary Entity Tables

• Notebook (notebook table): A container representing a research project or investigation.
• Source (source table): Raw artifacts such as PDFs, web pages, or audio files that have been ingested, chunked, and embedded for retrieval.
• Note (note table): User-generated or AI-generated commentary, often derived from or referencing specific sources.

Edge Types Defining Relationships

The following edge tables define how these entities interact:

Edge	Direction	Semantic Meaning
reference	Notebook → Source	Indicates that a notebook references a particular source document.
artifact	Notebook → Note	Signifies that a notebook contains a specific note.
refers_to	ChatSession → Notebook/Source	Links a chat session to its relevant notebook or source context.

Querying SurrealDB Graph Relationships

The Notebook model in open_notebook/domain/notebook.py encapsulates all graph traversal logic behind async helper methods. These methods use SurrealQL edge operators (->, <-) to walk relationships efficiently.

Retrieving Sources via the Reference Edge

The get_sources method traverses the reference edge from a notebook to its connected sources. It optionally omits the full_text field for performance when only metadata is required.

srcs = await repo_query(
    f"""
    select *{source_projection} from (
        select in as source from reference where out=$id
        fetch source
    ) order by source.updated desc
    """,
    {"id": ensure_record_id(self.id)},
)

This query selects all inbound records (sources) connected via the reference edge where the notebook is the outbound node (out=$id). The fetch source directive eagerly loads the full source record rather than just the ID.

Fetching Notes via the Artifact Edge

Similarly, the get_notes method walks the artifact edge to retrieve notes attached to the notebook. By default, it strips vector embeddings unless include_content=True is specified.

srcs = await repo_query(
    f"""
    select *{note_projection} from (
        select in as note from artifact where out=$id
        fetch note
    ) order by note.updated desc
    """,
    {"id": ensure_record_id(self.id)},
)

Both methods demonstrate how Open Notebook abstracts SurrealDB's graph syntax into domain-specific APIs, keeping the rest of the codebase agnostic to the underlying query language.

Cascade Control and Deletion Logic

The graph model enables sophisticated deletion strategies that distinguish between exclusive and shared resources. Before removing a notebook, the get_delete_preview method runs aggregation queries to assess impact.

Deletion Preview Queries

The following SurrealQL statements, implemented in open_notebook/domain/notebook.py, count dependent notes and evaluate source exclusivity:

-- Count notes attached to this notebook via artifact edges
SELECT count() as count FROM artifact WHERE out = $notebook_id GROUP ALL;

-- Check if sources are exclusive to this notebook or shared with others
SELECT
    id,
    count(->reference[WHERE out != $notebook_id].out) as assigned_others
FROM (SELECT VALUE <-reference.in AS sources FROM $notebook_id)[0]

The second query uses the <-reference.in operator to fetch inbound sources, then counts outgoing reference edges to other notebooks. If assigned_others is zero, the source is exclusive and safe to delete; otherwise, it should only be unlinked.

Executing Deletion with Cascade Options

The delete method uses this preview data toconditionally remove sources:

• Exclusive sources: Deleted entirely when delete_exclusive_sources=True.
• Shared sources: Unlinked (edge removed) but preserved for other notebooks.

This prevents accidental data loss while maintaining referential integrity without foreign key constraints.

Benefits of the Graph Model Architecture

Using SurrealDB's native graph capabilities provides three architectural advantages over traditional relational schemas:

• Bidirectional navigation: Edges enable fast lookups from notebook → source, source → notebook, and source → notes without complex JOINs or recursive CTEs.
• Fine-grained cascade control: The explicit edge table allows the deletion logic to analyze relationship cardinality and make context-aware decisions about unlinking versus deleting.
• Semantic search integration: Both text search (fn::text_search) and vector search (fn::vector_search) return record IDs that resolve directly back to graph nodes, maintaining relationship context in search results.

Practical Workflow: Traversing the Graph

The following async example demonstrates loading a notebook, listing its relationships, and executing a conditional deletion. This pattern mirrors the implementation in api/notebook_service.py.

import asyncio
from open_notebook.domain.notebook import Notebook

async def demo_graph_traversal(notebook_id: str):
    # Load the notebook domain model

    nb = await Notebook.get(notebook_id)
    
    # Retrieve all sources with full text content

    sources = await nb.get_sources(include_full_text=True)
    for src in sources:
        print(f"Source: {src.title}")
        print(f"Preview: {src.full_text[:200]}...\n")
    
    # Fetch notes without heavy embeddings

    notes = await nb.get_notes(include_content=False)
    for note in notes:
        preview = note.content[:100] if note.content else "[No content]"
        print(f"Note: {note.title} -> {preview}")
    
    # Analyze deletion impact

    preview = await nb.get_delete_preview()
    print(f"\nDeletion impact: {preview.notes_count} notes, "
          f"{preview.exclusive_sources_count} exclusive sources")
    
    # Delete notebook and orphan exclusive sources

    result = await nb.delete(delete_exclusive_sources=True)
    print(f"Deletion complete: {result}")

# Execute with a valid SurrealDB RecordID

# asyncio.run(demo_graph_traversal("notebook:018f3e4c..."))

The graph_utils.py module provides additional helpers for constructing these queries, while async_migrate.py handles schema definitions for tables and edges on application startup.

Summary

• SurrealDB edges (reference, artifact) replace foreign keys, enabling efficient graph traversal between notebooks, sources, and notes.
• Domain models in open_notebook/domain/notebook.py encapsulate SurrealQL queries, exposing clean Python APIs for graph operations.
• Deletion safety is enforced through preview queries that distinguish exclusive sources from shared ones, preventing accidental data loss.
• Bidirectional queries support both notebook-centric workflows (listing a notebook's sources) and source-centric workflows (finding which notebooks reference a source).

Frequently Asked Questions

How does Open Notebook represent many-to-many relationships between notebooks and sources?

Open Notebook uses the reference edge table to create a native many-to-many relationship. Rather than storing a foreign key in either table, the edge record links a notebook ID to a source ID, allowing a single source to be referenced by multiple notebooks and vice versa. This is queried using SurrealQL edge notation like select in as source from reference where out=$notebook_id.

What happens to source files when a notebook is deleted?

The deletion behavior depends on source exclusivity. The get_delete_preview method checks if sources have outgoing reference edges to other notebooks. Exclusive sources (linked only to the current notebook) can be permanently deleted by passing delete_exclusive_sources=True to the delete method, while shared sources are simply unlinked by removing the edge record, preserving them for other notebooks.

Can applications query backwards from a source to find its parent notebooks?

Yes. Because SurrealDB edges are bidirectional, you can traverse the inverse relationship using the <- operator. For example, SELECT VALUE <-reference.out FROM $source_id returns all notebooks that reference a given source. The domain models abstract this capability, though the underlying graph structure supports arbitrary traversal patterns.

Where is the SurrealDB schema and migration logic defined?

Schema definitions and edge table configurations reside in open_notebook/database/async_migrate.py, which executes on application startup to ensure tables (notebook, source, note) and edge tables (reference, artifact) exist. Helper utilities for building graph queries are located in open_notebook/utils/graph_utils.py, providing reusable functions for edge traversal and record resolution across the domain layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}