Open Notebook Communication Flow: How Frontend, API, and SurrealDB Interact

Open Notebook implements a three-tier architecture where the React/Next.js frontend communicates via HTTP/REST with FastAPI endpoints, which delegate database operations to an async repository layer that executes SurrealQL queries against SurrealDB using the AsyncSurreal client.

The lfnovo/open-notebook repository organizes its codebase into distinct layers that maintain strict separation of concerns. Understanding the communication flow between these layers reveals how the application handles asynchronous data persistence while keeping the API code completely decoupled from SurrealDB-specific implementations.

Frontend to API Communication

The frontend layer consists of React components built with Next.js that consume RESTful endpoints exposed by the backend.

HTTP Requests with TanStack Query

The UI issues standard HTTP requests (GET, POST, PUT, DELETE) to /api/* endpoints. While the application uses TanStack Query (formerly React Query) to manage server state, the underlying mechanism is standard fetch calls to the FastAPI backend.

// src/components/notebooks/NotebooksList.tsx
import { useQuery } from '@tanstack/react-query';

export const NotebooksList = () => {
  const { data, isLoading, error } = useQuery(['notebooks'], async () => {
    const res = await fetch('/api/notebooks');           // → FastAPI endpoint
    if (!res.ok) throw new Error('Failed to load notebooks');
    return res.json();                                   // JSON → UI
  });

  // render UI …
};

This approach ensures the frontend remains agnostic to the database technology, treating the FastAPI layer as the single source of truth for all data operations.

API Layer and Routing

The FastAPI backend handles HTTP request validation and delegates business logic to the repository tier.

FastAPI Router Structure

According to the source code in api/routers/notebooks.py, endpoints are organized using APIRouter objects that define HTTP methods and response models. These routers construct SurrealQL queries but do not execute them directly; instead, they invoke repository helpers.

# api/routers/notebooks.py

@router.get("/notebooks", response_model=List[NotebookResponse])
async def get_notebooks(...):
    # build SurrealQL query

    query = """
        SELECT *,
        count(<-reference.in) as source_count,
        count(<-artifact.in) as note_count
        FROM notebook
        ORDER BY {validated_order_by}
    """
    result = await repo_query(query)                     # ← repository layer

    # convert to Pydantic models → JSON response

    return [NotebookResponse(... ) for nb in result]

This architecture isolates HTTP concerns from database syntax, allowing the API layer to focus on request validation and response serialization while the repository handles SurrealDB specifics.

Repository Abstraction Layer

The open_notebook/database/repository.py file contains the critical bridge between Python async code and SurrealDB's graph database engine.

Async SurrealDB Connection Management

The repository maintains a single async connection pattern using AsyncSurreal from the surrealdb package. The db_connection function acts as an async context manager that handles authentication, namespace selection, and connection cleanup.

# open_notebook/database/repository.py

@asynccontextmanager
async def db_connection():
    db = AsyncSurreal(get_database_url())
    await db.signin({"username": os.getenv("SURREAL_USER"), "password": get_database_password()})
    await db.use(get_database_namespace(), get_database_name())
    try:
        yield db
    finally:
        await db.close()

SurrealQL Execution and Normalization

The repository exposes CRUD helpers such as repo_query, repo_create, repo_update, and repo_delete. These functions build SurrealQL strings and execute them via the async connection, then normalize results using parse_record_ids to convert SurrealDB's record IDs into plain JSON objects suitable for API responses.

# open_notebook/database/repository.py

async def repo_query(query_str: str, vars: Optional[Dict[str, Any]] = None) -> List[Dict[str, Any]]:
    async with db_connection() as connection:            # ← AsyncSurreal client

        result = parse_record_ids(
            await connection.query(query_str, vars)      # ← SurrealQL execution

        )
        if isinstance(result, str):
            raise RuntimeError(result)                    # error handling

        return result

This normalization step ensures that the API receives standard Python dictionaries regardless of SurrealDB's internal graph record formats.

Complete Request Flow Example

Tracing a single notebook retrieval request demonstrates the full communication path:

1. Frontend: The NotebooksList component calls fetch('/api/notebooks') via TanStack Query.
2. API Router: The get_notebooks function in api/routers/notebooks.py receives the HTTP GET request and constructs a SurrealQL query with graph traversals (count(<-reference.in)).
3. Repository: The router awaits repo_query(), which opens an async connection via db_connection(), signs in to SurrealDB, and executes the query using connection.query().
4. Database: SurrealDB processes the SurrealQL, performs the graph traversal counts, and returns raw records.
5. Normalization: parse_record_ids transforms the database response into clean dictionaries.
6. Response: The router converts the dictionaries to Pydantic models (NotebookResponse) and returns JSON to the frontend.

This flow ensures that SurrealDB-specific logic never leaks into the frontend or API route handlers, maintaining clean architectural boundaries while leveraging SurrealDB's graph capabilities for complex queries like count(<-reference.in).

Summary

• Open Notebook uses a three-tier architecture separating React frontend, FastAPI backend, and SurrealDB data store.
• HTTP/REST serves as the communication protocol between frontend and API, with TanStack Query managing server state in the UI.
• Repository pattern in open_notebook/database/repository.py abstracts all SurrealDB interactions behind async helpers like repo_query and repo_create.
• AsyncSurreal client maintains persistent connections using db_connection() context managers, ensuring efficient database access.
• Data normalization via parse_record_ids converts SurrealDB graph records to plain JSON before API serialization.

Frequently Asked Questions

How does the frontend communicate with the API in Open Notebook?

The React/Next.js frontend communicates via standard HTTP requests to /api/* endpoints exposed by FastAPI. The codebase uses TanStack Query to manage caching and loading states, but fundamentally relies on fetch calls that send JSON payloads and receive JSON responses, keeping the frontend decoupled from database implementation details.

What is the role of the repository layer in Open Notebook's architecture?

The repository layer in open_notebook/database/repository.py acts as a database abstraction barrier. It encapsulates all SurrealDB-specific logic—including connection management via db_connection(), query execution, and result normalization through parse_record_ids—allowing FastAPI routers to work with plain Python dictionaries rather than SurrealDB record objects.

How does SurrealDB handle queries from the Open Notebook API?

The API sends SurrealQL strings through the AsyncSurreal client using the connection.query() method. The repository helpers construct these queries dynamically, including graph traversals like count(<-reference.in) for relationship counting, and execute them within async context managers that handle authentication and connection lifecycle automatically.

Why does Open Notebook use async connections to SurrealDB?

Open Notebook uses async connections via AsyncSurreal and Python's asynccontextmanager to prevent blocking the FastAPI event loop during database operations. This approach allows the API to handle multiple concurrent requests efficiently while maintaining proper connection hygiene through automatic cleanup in the finally block of db_connection().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}