Three-Tier Architecture Communication in Open Notebook: Frontend, API, and SurrealDB Explained

Open Notebook implements async-first three-tier architecture communication where the Next.js frontend exchanges data with FastAPI via HTTP REST, while the API layer queries SurrealDB through an asynchronous repository pattern supporting graph relationships and vector search.

Open Notebook (lfnovo/open-notebook) separates concerns across three distinct tiers: a React-based presentation layer, a Python orchestration service, and a graph-vector database. This architecture ensures that UI rendering, business logic validation, and data persistence remain decoupled, with each layer communicating through well-defined, typed contracts.

Architecture Overview

The system stack consists of three specialized layers documented in docs/7-DEVELOPMENT/architecture.md. Each tier operates independently and communicates asynchronously to prevent blocking operations.

• Frontend Tier: Next.js 15 with React 19 (TypeScript) handles UI rendering and user interactions. It communicates with the backend via HTTP requests managed by TanStack Query.
• API Tier: FastAPI 0.104+ running on Python 3.11 exposes RESTful endpoints, validates payloads with Pydantic, and coordinates domain logic including LangGraph workflows.
• Database Tier: SurrealDB serves as the persistent store, providing ACID transactions, native graph traversal, and vector similarity search for embeddings.

Frontend to API Communication

The frontend uses TanStack Query (React Query) to manage server state and issue asynchronous HTTP requests to the FastAPI server, typically at http://localhost:5055. This pattern ensures cached data remains synchronized while handling loading and error states automatically.

In src/hooks/useNotebooks.ts, the application fetches notebook data using a standard fetch pattern wrapped in useQuery:

import { useQuery } from '@tanstack/react-query';

export const useNotebooks = () =>
  useQuery(['notebooks'], async () => {
    const resp = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/notebooks`);
    if (!resp.ok) throw new Error('Failed to load notebooks');
    return resp.json(); // → NotebookResponse[]
  });

The request hits the /notebooks endpoint defined in api/routers/notebooks.py, which returns a JSON array validated against Pydantic models.

API to SurrealDB Communication

The FastAPI layer delegates all database operations to a centralized repository layer located in open_notebook/database/repository.py. This abstraction ensures that SurrealDB-specific logic remains isolated from business rules, and all I/O operations remain asynchronous to prevent blocking the event loop.

The core helper repo_query opens a temporary database connection, executes SurrealQL, and normalizes RecordID objects to plain strings:

# open_notebook/database/repository.py

async def repo_query(
    query_str: str, 
    vars: Optional[Dict[str, Any]] = None
) -> List[Dict[str, Any]]:
    async with db_connection() as connection:
        result = parse_record_ids(await connection.query(query_str, vars))
        if isinstance(result, str):
            raise RuntimeError(result)
        return result

API endpoints in api/routers/notebooks.py utilize this helper to execute complex queries with graph traversal and aggregation:

# api/routers/notebooks.py

@router.get("/notebooks", response_model=List[NotebookResponse])
async def get_notebooks(...):
    query = """
        SELECT *,
        count(<-reference.in) as source_count,
        count(<-artifact.in) as note_count
        FROM notebook
        ORDER BY {validated_order_by}
    """
    result = await repo_query(query)
    return [
        NotebookResponse(
            id=str(nb.get("id", "")),
            name=nb.get("name", ""),
            source_count=nb.get("source_count", 0),
            note_count=nb.get("note_count", 0),
        )
        for nb in result
    ]

SurrealDB Schema and Graph Relationships

SurrealDB stores entities in tables (e.g., notebook, source, note) and uses graph edges to model many-to-many relationships without foreign key constraints. The schema defines edges such as reference (linking sources to notebooks) and artifact (linking notes to notebooks).

The data model follows this structure:

Notebook ── reference ──► Source
      │
      └─ artifact ──► Note

Schema migrations stored in /migrations/*.surql apply automatically when the API starts, ensuring consistency across environments. The database handles both structured queries and vector similarity searches through SurrealQL functions like vector::similarity (denoted as <=> in queries).

End-to-End Communication Flow

Consider the workflow for adding a source to a notebook, which demonstrates how all three tiers coordinate:

1. Frontend: The UI invokes POST /notebooks/{nb_id}/sources/{src_id} using TanStack Query’s mutation hook.
2. API Layer: The FastAPI router validates both IDs, verifies existence, and executes a graph relation.
3. Database Layer: The repository executes RELATE to create the edge.
4. Frontend Update: TanStack Query invalidates the cache, triggering a refetch that includes the updated source_count.

The API endpoint implementation in api/routers/notebooks.py shows the exact coordination:

@router.post("/notebooks/{notebook_id}/sources/{source_id}")
async def add_source_to_notebook(notebook_id: str, source_id: str):
    await Notebook.get(notebook_id)          # verify notebook

    await Source.get(source_id)              # verify source

    
    await repo_query(
        "RELATE $source_id->reference->$notebook_id",
        {
            "notebook_id": ensure_record_id(notebook_id),
            "source_id":   ensure_record_id(source_id),
        },
    )
    return {"message": "Source linked to notebook successfully"}

The ensure_record_id helper converts string identifiers into SurrealDB record IDs (e.g., notebook:123) before parameterization, preventing injection vulnerabilities.

Summary

• Three-tier separation in Open Notebook uses Next.js, FastAPI, and SurrealDB with clean async boundaries between each layer.
• Frontend communication relies on TanStack Query to manage HTTP REST calls to the FastAPI server, ensuring type-safe data fetching.
• API-to-database communication uses the repo_query repository pattern in open_notebook/database/repository.py, which handles connection pooling, query execution, and RecordID normalization.
• SurrealDB acts as both a graph database (using RELATE for edges) and a vector store, accessed through SurrealQL with full ACID compliance.
• Async-first design ensures that database operations never block the FastAPI event loop, maintaining high concurrency for LangGraph workflow execution.

Frequently Asked Questions

How does the frontend handle asynchronous data fetching?

The frontend uses TanStack Query to wrap standard fetch calls, providing automatic caching, background refetching, and error handling. Hooks like useNotebooks encapsulate the HTTP logic, ensuring components remain declarative while communicating with the FastAPI endpoints at NEXT_PUBLIC_API_URL.

What is the repository pattern used in the API layer?

According to open_notebook/database/repository.py, the repository pattern abstracts SurrealDB-specific implementation details behind generic functions like repo_query and repo_create. This allows FastAPI routers to execute complex SurrealQL statements without managing connection lifecycles or RecordID conversions directly.

How does SurrealDB manage relationships between notebooks and sources?

SurrealDB uses graph edges rather than foreign keys. The API creates relationships using the RELATE statement (e.g., RELATE $source_id->reference->$notebook_id), which creates a directional edge that can be traversed with queries like count(<-reference.in) to aggregate related data efficiently.

Is the communication between the API and database synchronous?

No. All database communication in Open Notebook is asynchronous. The repo_query function uses async with db_connection() to acquire temporary connections, and FastAPI endpoints are defined with async def, ensuring that database I/O does not block the event loop during concurrent LangGraph workflow executions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}