Open Notebook Three-Tier Architecture: Frontend, FastAPI, and SurrealDB Explained

Open Notebook implements a strict three-tier separation where a React/Next.js frontend communicates via HTTP with a FastAPI orchestration layer, which persists data in SurrealDB using an async repository pattern.

The lfnovo/open-notebook project organizes its codebase into distinct layers that isolate UI concerns from AI workflow orchestration and data persistence. This three-tier architecture cleanly separates the Frontend (React/Next.js), API (FastAPI), and Database (SurrealDB) to enable independent scaling and maintainable LangGraph integration. Understanding these boundaries is essential for contributing to the codebase or extending its semantic search capabilities.

Architecture Overview

The system divides responsibilities across three independent tiers, with the frontend operating on port 3000, FastAPI on port 5055, and SurrealDB on port 8000.

Frontend Layer: React and Next.js

The presentation tier is built with Next.js (TypeScript) and styled using Tailwind CSS with Shadcn UI components. State management relies on Zustand for client-side stores, while TanStack Query (React Query) handles server-state synchronization, caching, and background refetching. This layer is responsible for rendering notebooks, sources, chat interfaces, and search UIs, but performs no direct database access. All data operations flow through HTTP requests to the FastAPI backend, typically targeting http://localhost:5055.

API Layer: FastAPI and LangGraph

The middle tier runs FastAPI ≥ 0.104 with Pydantic v2 models for request validation. This layer serves as the orchestration hub, exposing REST endpoints for CRUD operations while managing complex LangGraph workflows for content ingestion, semantic search, and transformations. The API uses Esperanto for AI-provider abstraction and handles asynchronous job queues (such as podcast generation) via the surreal-commands system. In api/main.py, the FastAPI application configures CORS and mounts routers that delegate to service modules like api/notebook_service.py and api/sources_service.py.

Data Layer: SurrealDB

The persistence tier uses SurrealDB as a graph database storing structured records (Notebook, Source, Note, ChatSession, Credential) along with their relationships and vector embeddings for semantic search. Access is restricted to the FastAPI layer through an async driver wrapper defined in open_notebook/database/repository.py. Schema migrations run automatically at API startup via open_notebook/database/async_migrate.py, ensuring the database structure remains synchronized with the application code.

Data Flow Between Tiers

The communication pattern follows a strict request-response cycle with support for asynchronous job processing:

1. The UI issues an HTTP request (e.g., GET /api/notebooks) to the FastAPI router in api/routers/notebooks.py.
2. FastAPI delegates to a service layer (api/notebook_service.py) which uses the repository pattern (open_notebook/database/repository.py) to query SurrealDB.
3. SurrealDB returns graph records; FastAPI converts them to Pydantic models and returns JSON to the client.
4. For long-running tasks (e.g., podcast generation), FastAPI enqueues a job via surreal-commands, returning a command ID immediately while the UI polls /api/commands/{id} for status updates.

Key Implementation Files

• frontend/app/page.tsx — Next.js entry point that bootstraps the UI and configures the base API URL.
• api/main.py — FastAPI application setup, CORS configuration, lifespan hooks, and router mounting.
• api/notebook_service.py — Business logic implementation for notebook CRUD operations.
• open_notebook/database/repository.py — Async SurrealDB driver wrapper managing connections and low-level queries.
• open_notebook/database/async_migrate.py — Automatic schema migration runner executed at API startup.
• api/podcast_service.py — Async job submission logic for podcast generation via surreal-commands.

Implementation Examples

Fetching Notebooks from the Frontend

The frontend uses TanStack Query to fetch notebook data from the FastAPI endpoint:

import { useQuery } from '@tanstack/react-query';
import axios from 'axios';

type Notebook = {
  id: string;
  title: string;
  created_at: string;
};

export const useNotebooks = () => {
  return useQuery<Notebook[]>(['notebooks'], async () => {
    const { data } = await axios.get('/api/notebooks');
    return data;
  });
};

The endpoint /api/notebooks is defined in api/notebook_service.py and exposed through api/routers/notebooks.py.

Creating Records via the API Layer

TypeScript interfaces define the contract with the FastAPI backend:

import axios from 'axios';

export const createSource = async (payload: {
  notebook_id: string;
  url: string;
}) => {
  const { data } = await axios.post('/api/sources', payload);
  return data; // Returns the created Source record
};

FastAPI receives this request in api/sources_service.py and persists the record via open_notebook/database/repository.py.

Querying SurrealDB from the Repository

The repository layer abstracts direct database access using the async SurrealDB driver:

from open_notebook.database.repository import get_surreal

async def list_notebooks():
    db = await get_surreal()
    result = await db.query("SELECT * FROM notebook;")
    return result

The get_surreal() function reads the SURREAL_URL environment variable and manages connection pooling.

Handling Async Operations

For long-running AI tasks, the frontend submits jobs and polls for completion:

import axios from 'axios';

export const startPodcast = async (notebookId: string) => {
  const { data } = await axios.post(`/api/podcasts/${notebookId}/generate`);
  return data.command_id;
};

export const pollCommand = async (commandId: string) => {
  const { data } = await axios.get(`/api/commands/${commandId}`);
  return data; // { status: "running" | "completed" | "failed", result?: … }
};

The api/podcast_service.py handler submits a surreal-commands job and returns the command ID, while api/routers/commands.py provides the polling endpoint.

Summary

• Three-tier separation ensures the React frontend remains lightweight while FastAPI handles complex AI orchestration and SurrealDB manages graph-structured data.
• Repository pattern in open_notebook/database/repository.py isolates database-specific code from business logic.
• Async job processing via surreal-commands prevents HTTP timeouts during long-running operations like podcast generation.
• Automatic migrations execute at startup via async_migrate.py, keeping schema changes synchronized with the API deployment.

Frequently Asked Questions

How does the frontend communicate with the FastAPI backend?

The frontend communicates via standard HTTP requests to http://localhost:5055, using TanStack Query for data fetching, caching, and mutations. Zustand handles purely client-side state, while server state never touches the UI layer directly.

What role does SurrealDB play in the three-tier architecture?

SurrealDB serves as the persistent graph database storing records (Notebooks, Sources, Notes), vector embeddings for semantic search, and relationship mappings. It is accessed exclusively by the FastAPI layer through the async repository pattern, ensuring the frontend has no database credentials or direct connection capabilities.

How does Open Notebook handle long-running AI tasks?

Long-running operations use the surreal-commands queue system. FastAPI submits jobs to this queue and immediately returns a command ID to the frontend. The UI polls the /api/commands/{id} endpoint for status updates, allowing the API to process computationally expensive tasks (like podcast generation via LangGraph) without blocking HTTP connections.

Where are database schema migrations managed?

Schema migrations are defined in open_notebook/database/async_migrate.py and execute automatically during FastAPI startup via the lifespan hook in api/main.py. This ensures the SurrealDB schema is always synchronized with the current application code before the API begins accepting requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}