# How content-core Extracts Content from PDFs, Videos, Audio, and URLs in Open Notebook

> Learn how content-core extracts text from PDFs, videos, audio, and URLs. Discover its specialized engines for efficient content processing in Open Notebook.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-15

---

**The `content-core` library processes raw files and URLs by inspecting the `ProcessSourceState` configuration, automatically selecting specialized engines like `pdfminer` for documents, `ffmpeg` with speech-to-text models for media, and `readability-lxml` for web pages.**

The `content-core` library serves as the extraction engine for the Open Notebook project, handling the transformation of unstructured data into clean markdown. When users upload PDFs, audio files, YouTube links, or web URLs, the system constructs a `ProcessSourceState` object that dictates which parsing pipeline to execute. This architecture allows Open Notebook to support over 50 file formats through a unified asynchronous interface.

## The ProcessSourceState Abstraction

At the heart of the extraction workflow is the `ProcessSourceState` object defined in `content_core.common`. This dictionary-like structure tells `content-core` which extraction engine to use and supplies optional model configurations for speech-to-text processing.

In [`open_notebook/graphs/source.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/source.py), Open Notebook prepares this state and calls the extraction function:

```python
from content_core import extract_content
from content_core.common import ProcessSourceState

# Build the state object based on user input

state: ProcessSourceState = {
    "document_engine": "auto",      # or "pdfminer", "docx2txt", etc.

    "url_engine": "auto",           # or "readability"

    "output_format": "markdown",
    "file_path": "/path/to/file.pdf",
    "url": None,
    "audio_provider": "openai",     # optional STT configuration

    "audio_model": "whisper-1",
}

# Execute extraction

processed = await extract_content(state)

```

The `extract_content` function inspects this state to determine whether the source is a local file or remote URL, then routes to the appropriate engine based on file extension or MIME type.

## PDF and Office Document Extraction

For document files, `content-core` selects parsers based on the `document_engine` field. When set to `"auto"`, the library automatically detects the format and chooses the appropriate backend.

- **PDF files**: Uses `pdfminer.six` to extract text and metadata (title, author)
- **DOCX files**: Routes through `docx2txt` to convert Word documents to plain text
- **ODT/Office files**: Utilizes `odfpy` for OpenDocument formats

The engine streams the file through the selected parser and returns structured content along with extracted metadata. This process is triggered when `file_path` points to a local document and `document_engine` specifies the parser or is set to `"auto"`.

## Audio and Video Processing

Media processing follows a two-stage pipeline involving audio extraction and speech-to-text transcription.

For **audio files** (MP3, WAV, etc.):
1. `ffmpeg` normalizes the audio to 16kHz mono PCM format
2. The configured speech-to-text provider transcribes the audio (default: OpenAI Whisper)
3. The resulting transcript populates the `content` field of the returned state

For **video files** (MP4, YouTube URLs):
1. The system first attempts to fetch **closed captions** via the YouTube Data API
2. If no captions exist, `ffmpeg` extracts the audio track from the downloaded video
3. The audio pipeline then transcribes the extracted audio using the configured model

The `audio_provider` and `audio_model` fields in `ProcessSourceState` allow customization of the transcription backend, supporting OpenAI's Whisper API or local models via Ollama.

## Web Page and URL Extraction

For web sources, `content-core` employs a readability-focused approach to extract article content from HTML boilerplate.

When processing a URL:
1. The library downloads the raw HTML using `httpx` (with support for SOCKS proxies)
2. The HTML passes through `readability-lxml` to strip navigation, ads, and other non-content elements
3. The cleaned article converts to Markdown using `html2text` internally

The `url_engine` field controls this behavior—setting it to `"auto"` enables the readability pipeline, while specific engines can be selected for alternative parsing strategies.

## Engine Selection and Auto-Detection

Open Notebook configures default engines in [`open_notebook/domain/content_settings.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/content_settings.py). When fields are set to `"auto"`, `content-core` implements the following fallback logic:

- **PDF/DOCX/ODT** → `pdfminer`, `docx2txt`, or `odfpy` based on file extension
- **HTML/URLs** → `beautifulsoup4` + `readability-lxml` → Markdown conversion
- **Audio** → Selected speech-to-text model (OpenAI Whisper by default)
- **Video** → YouTube caption fetch → audio extraction → speech-to-text

This registry-based design means adding support for new file formats requires updating only the `content-core` library, without modifying Open Notebook's source code.

## Code Examples

### Extracting Text from a Local PDF

```python
from content_core import extract_content
from content_core.common import ProcessSourceState

state: ProcessSourceState = {
    "document_engine": "auto",
    "output_format": "markdown",
    "file_path": "/tmp/research_paper.pdf",
    "url": None,
}

processed = await extract_content(state)
print(processed.content)   # Extracted markdown text

print(processed.title)   # Title from PDF metadata

```

### Processing a YouTube Video with Caption Fallback

```python
state = {
    "url_engine": "auto",
    "document_engine": "auto",
    "output_format": "markdown",
    "url": "https://www.youtube.com/watch?v=abc123",
    "audio_provider": "openai",
    "audio_model": "whisper-1",
}

processed = await extract_content(state)
print(processed.content)   # Transcript from captions or transcribed audio

```

### Transcribing an Audio File with a Local Model

```python
state = {
    "document_engine": "auto",
    "output_format": "markdown",
    "file_path": "/tmp/interview.mp3",
    "audio_provider": "ollama",
    "audio_model": "whisper.cpp",
}

processed = await extract_content(state)
print(processed.content)   # Transcribed speech content

```

### Extracting Article Content from a Web URL

```python
state = {
    "url_engine": "auto",
    "document_engine": "auto",
    "output_format": "markdown",
    "url": "https://example.com/blog/article",
}

processed = await extract_content(state)
print(processed.content)   # Clean article body in markdown format

```

## Summary

- **`content-core`** handles all content extraction in Open Notebook through a unified `extract_content` function
- **`ProcessSourceState`** objects configure the extraction pipeline, specifying engines for documents, URLs, and audio processing
- **Document engines** like `pdfminer` and `docx2txt` handle PDFs and Office files, while `readability-lxml` processes web pages
- **Media processing** uses `ffmpeg` for audio extraction and supports configurable speech-to-text models including OpenAI Whisper
- **Auto-detection** allows the system to select appropriate parsers based on file extensions and MIME types without manual configuration

## Frequently Asked Questions

### What is ProcessSourceState and where is it defined?

`ProcessSourceState` is a typed dictionary defined in `content_core.common` that serves as the configuration contract between Open Notebook and the extraction library. It contains fields like `document_engine`, `url_engine`, `file_path`, and `audio_model` that tell `content-core` which parsing pipeline to execute and which specific models to use for transcription tasks.

### How does content-core handle YouTube videos without captions?

When processing YouTube URLs, `content-core` first attempts to retrieve official closed captions via the YouTube Data API. If no captions are available, the library downloads the video, extracts the audio track using `ffmpeg`, and passes that audio to the configured speech-to-text model (such as OpenAI's Whisper) to generate a transcript.

### Can I use a custom speech-to-text model instead of OpenAI Whisper?

Yes. The `audio_provider` and `audio_model` fields in `ProcessSourceState` allow you to specify alternative providers. For example, setting `"audio_provider": "ollama"` and `"audio_model": "whisper.cpp"` routes transcription through a locally-hosted model rather than the OpenAI API, enabling offline processing and privacy-sensitive workflows.

### Which engines are used for extracting text from PDFs?

For PDF documents, `content-core` defaults to `pdfminer.six`, a robust Python library that extracts text and metadata while preserving document structure. When `document_engine` is set to `"auto"`, the system automatically selects `pdfminer` for `.pdf` files, `docx2txt` for `.docx` files, and `odfpy` for OpenDocument formats based on file extension detection.