How Open Notebook Extracts and Embeds Content from PDFs, Videos, and URLs

Open Notebook uses a LangGraph workflow orchestrated in open_notebook/graphs/source.py to extract content via the content_core library, then persists the text to SurrealDB and generates vector embeddings through open_notebook/utils/embedding.py for semantic search.

Open Notebook, an open-source project under lfnovo/open-notebook, treats every inbound source—whether a PDF file, YouTube video, or web URL—as a content-ingestion job that runs through a structured pipeline. This system leverages a LangGraph workflow to automatically extract text, handle audio transcription for video content, and convert everything into searchable vector embeddings. Understanding how Open Notebook extracts and embeds content from PDFs, videos, and URLs reveals the architecture behind its privacy-first, AI-powered research capabilities.

The LangGraph Content Extraction Pipeline

Workflow Orchestration in source.py

The extraction logic resides in open_notebook/graphs/source.py, which defines a LangGraph workflow that processes sources through distinct nodes. The pipeline follows this sequence:

START → content_process → save_source → (optional) transform_content → END

The content_process Node

The workflow begins at the content_process node, which constructs a ProcessSourceState object from the content-core library. This state object configures extraction parameters, setting the output format to markdown and enabling automatic engine detection. When processing video URLs, the node checks for configured speech-to-text models and injects audio_provider and audio_model into the state to enable transcription of audio-only media.

Actual extraction occurs through a single call to content_core.extract_content(state). This external library handles format-specific parsing for PDFs, Office documents, images, plain-text files, and HTML pages. For video content, it retrieves available subtitles or falls back to the configured STT model to generate transcripts. Soft failures during extraction are converted to ValueError exceptions to mark jobs as retryable.

Persistence and Vectorization

Saving Sources to SurrealDB

After extraction, the save_source node persists results to the Source record in SurrealDB. The Asset field stores the original URL or local file path, while source.full_text receives the extracted markdown content. The system updates the title only when the extracted title is non-empty and the current title remains the placeholder "Processing…".

Generating Vector Embeddings

When the ingestion request includes embed=True, the save_source node triggers await source.vectorize(). Implemented in open_notebook/domain/notebook.py, this method delegates to open_notebook/utils/embedding.generate_embeddings.

The embedding utility chunks the text and sends it to the configured embedding model via the Esperanto abstraction layer. The resulting dense vectors are normalized and stored alongside the source record, enabling semantic search capabilities across the entire knowledge base.

Practical Implementation Examples

Ingesting PDFs via Multipart Upload

To extract and embed content from PDFs through the API:

import httpx

api = "http://localhost:5055"
pdf_path = "report.pdf"

with open(pdf_path, "rb") as f:
    files = {"file": ("report.pdf", f, "application/pdf")}
    data = {"embed": "true"}
    resp = httpx.post(f"{api}/sources", files=files, data=data)
    print(resp.json())

The POST /sources endpoint in api/sources_service.py forwards the file to the source_graph workflow.

Processing YouTube Videos

For video URLs, the pipeline handles audio extraction and transcription:

curl -X POST http://localhost:5055/sources \
     -H "Content-Type: application/json" \
     -d '{"url":"https://www.youtube.com/watch?v=abc123","embed":true}'

If a speech-to-text model is configured, Open Notebook downloads the audio stream, runs the model, and stores the generated transcript as source.full_text.

Scraping Web URLs

Generic web pages follow the same pattern:

import httpx

resp = httpx.post(
    "http://localhost:5055/sources",
    json={"url": "https://example.com/article", "embed": True},
)
print(resp.json())

The content_core library selects the appropriate HTML scraper to convert the page to markdown.

Querying Embedded Content

Once vectorized, content is searchable via nearest-neighbor lookup:

import httpx

query = "machine learning trends 2025"
resp = httpx.post(
    "http://localhost:5055/search",
    json={"query": query, "k": 5}
)
print(resp.json()["results"])

Summary

• Open Notebook uses a LangGraph workflow in open_notebook/graphs/source.py to orchestrate content ingestion across PDFs, videos, and URLs.
• The content_process node leverages the content_core library to extract text, with special handling for video transcription via speech-to-text models.
• Extracted content is persisted to SurrealDB through the save_source node, which updates the Source domain model.
• Vector embeddings are generated via open_notebook/utils/embedding.py using the Esperanto abstraction, enabling semantic search across all ingested sources.
• The API entry point at api/sources_service.py provides HTTP access to the entire pipeline.

Frequently Asked Questions

What file formats does Open Notebook support for content extraction?

According to the source code in open_notebook/graphs/source.py, Open Notebook delegates extraction to the content_core library, which handles PDFs, Office documents, plain-text files, images, and web URLs. For video content, it processes YouTube and other video URLs by extracting subtitles or transcribing audio using configured speech-to-text models.

How does Open Notebook handle video content without subtitles?

When processing video URLs, the content_process node checks for configured audio_provider and audio_model settings. If speech-to-text is configured, the system downloads the audio stream and runs the STT model to generate a transcript, storing the result as markdown in source.full_text.

Where are the vector embeddings stored in Open Notebook?

The Source.vectorize() method in open_notebook/domain/notebook.py generates embeddings through open_notebook/utils/embedding.py and stores the resulting dense vectors directly in SurrealDB alongside the source record. These vectors power the semantic search functionality accessible via the /search API endpoint.

Can the content extraction pipeline be retried if it fails?

Yes. The workflow in open_notebook/graphs/source.py converts soft extraction failures into ValueError exceptions, which mark the job as retryable. This ensures that transient failures such as network issues or temporary service unavailability don't permanently corrupt the source record.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}