What Is the Primary Purpose of the Open Notebook Project?

Open Notebook is an open-source, privacy-first AI-powered research assistant designed to let users upload, process, search, and interact with multi-modal content such as PDFs, audio, video, and web pages while operating entirely under their own control via local or self-hosted deployment.

The lfnovo/open-notebook repository delivers a comprehensive solution for researchers and knowledge workers who need intelligent content analysis without sacrificing data privacy. Unlike cloud-dependent SaaS platforms, Open Notebook leverages SurrealDB for vector storage and the Esperanto library for multi-provider LLM support, allowing you to run advanced AI workflows on your own infrastructure. This article examines the project's core purpose by exploring its architecture, key capabilities, and implementation details found in the source code.

Core Capabilities That Define the Project's Purpose

Multi-Modal Content Ingestion

At the heart of Open Notebook lies the ability to process diverse content types through a unified pipeline. The ingestion workflow, defined in open_notebook/graphs/source.py, uses a LangGraph state machine to handle extraction, embedding, and storage. When you upload a PDF, audio file, or video, the system automatically extracts text, generates vector embeddings, and indexes the content for semantic retrieval.

Semantic Search with Vector Embeddings

The project implements vector-based search through SurrealDB, enabling context-aware lookup across all stored materials. This functionality is managed by the database abstraction layer in open_notebook/database/repo.py, which provides an async repository pattern for notebooks, sources, and notes. The vector search capability allows users to find relevant information based on conceptual meaning rather than exact keyword matches.

Conversational AI Interface

Open Notebook provides a chat interface powered by multiple LLM providers via the model manager in open_notebook/ai/model_manager.py. This unified interface supports OpenAI, Anthropic, Ollama, and other providers through the Esperanto library, allowing users to query their personal data using natural language. The chat endpoint calls LangGraph workflows to pull relevant sources and synthesize contextual responses.

Automated Podcast Generation

A distinctive feature is the ability to convert notes and chat transcripts into polished audio podcasts. The asynchronous job processing is handled by open_notebook/podcasts/service.py, which queues text-to-speech tasks and packages the output as downloadable audio files. This transforms static research notes into consumable audio content.

Technical Architecture and Implementation

The FastAPI entry point in api/main.py configures the application and registers routers for the REST API, while the frontend synchronizes state through Zustand stores in frontend/src/lib/stores/useNotebook.ts. This architecture ensures a seamless connection between the user interface and backend services.

The system achieves its privacy-first mandate by running SurrealDB as the primary database for both structured data and vector embeddings, eliminating the need for external vector databases. The Esperanto library abstraction in open_notebook/ai/model_manager.py decouples the application from specific AI providers, enabling users to switch between cloud APIs and local models like those served via Ollama.

Practical Examples: Using the Open Notebook API

The following examples demonstrate how to interact with the system via its REST API endpoints running on localhost:5055.

Uploading a PDF Source

import requests

API_URL = "http://localhost:5055"

files = {"file": open("research.pdf", "rb")}
data = {"title": "Quantum Computing Overview"}

resp = requests.post(f"{API_URL}/sources", files=files, data=data)
print(resp.json())

This request triggers the extraction and embedding workflow defined in open_notebook/graphs/source.py, processing the PDF into searchable content.

Performing Semantic Search

import requests

query = "What are the main challenges of quantum error correction?"
resp = requests.get(f"{API_URL}/search", params={"q": query})
results = resp.json()["hits"]

for hit in results[:3]:
    print(f"- {hit['title']}: {hit['snippet']}")

This leverages SurrealDB's built-in vector search capabilities to retrieve the most relevant notes based on semantic similarity.

Chatting with Your Data

import requests

payload = {
    "messages": [
        {"role": "user", "content": "Summarise the key points from the PDF I just uploaded."}
    ],
    "model": "gpt-4o-mini"
}

resp = requests.post(f"{API_URL}/chat", json=payload)
print(resp.json()["answer"])

The chat endpoint invokes the LangGraph workflow, retrieves relevant sources from the vector database, and synthesizes a response using the specified LLM provider.

Generating a Podcast

import requests

note_id = "note:12345"
resp = requests.post(f"{API_URL}/podcasts", json={"note_id": note_id})
job_id = resp.json()["job_id"]
print(f"Podcast generation started, job ID: {job_id}")

This submits an asynchronous job to open_notebook/podcasts/service.py, which converts the note's text to speech and packages it as an audio file.

Summary

• Open Notebook serves as a self-hosted, privacy-first AI research assistant that processes multi-modal content including PDFs, audio, video, and web pages.
• The architecture relies on SurrealDB for vector storage and LangGraph for workflow orchestration, with provider abstraction via the Esperanto library.
• Key components include the FastAPI entry point (api/main.py), ingestion workflows (open_notebook/graphs/source.py), and podcast services (open_notebook/podcasts/service.py).
• Users retain full data sovereignty by running the system locally or on private infrastructure, avoiding third-party cloud dependencies.

Frequently Asked Questions

What makes Open Notebook different from other AI note-taking apps?

Unlike commercial alternatives that store data on proprietary cloud servers, Open Notebook is designed for local or self-hosted deployment, giving users complete control over their information. The integration of SurrealDB for vector search and support for multiple LLM providers through the Esperanto library ensures flexibility without compromising privacy.

Can Open Notebook process audio and video files, or just text documents?

Open Notebook handles multi-modal content including PDFs, audio, video, and web pages. The ingestion pipeline in open_notebook/graphs/source.py automatically extracts and embeds content from these various formats, making them searchable and available for chat interactions.

How does the podcast generation feature work?

The podcast service in open_notebook/podcasts/service.py queues asynchronous text-to-speech jobs that convert notes or chat transcripts into audio files. Users can submit generation requests via the API, receive a job ID, and download the polished audio output once processing completes.

Is it possible to use local LLMs instead of OpenAI or Anthropic?

Yes. The open_notebook/ai/model_manager.py file implements a unified interface that supports Ollama and other local providers through the Esperanto library, allowing you to run the entire stack—including AI inference—on your own hardware without external API calls.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}