# How PyMuPDF Converts PDF Pages to Markdown for LLM Processing: A Technical Deep Dive

> Learn how PyMuPDF converts PDF pages to Markdown for LLM processing. Discover the eight-stage pipeline analyzing layout, text, tables, and images for LLM-ready documents.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: deep-dive
- Published: 2026-07-22

---

**PyMuPDF converts PDF pages to Markdown through an eight-stage pipeline that analyzes document layout, detects headers by font size, extracts formatted text with inline styling, renders tables as pipe-delimited Markdown, and optionally embeds images as Base64 to produce LLM-ready documents.**

The `interviewstreet/hiring-agent` repository implements a sophisticated PDF-to-Markdown conversion system using PyMuPDF that transforms unstructured PDF documents into structured, GitHub-flavored Markdown optimized for large language model ingestion. This technical deep-dive examines the [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py) implementation to reveal exactly how PyMuPDF converts PDF pages to Markdown for LLM processing while preserving document hierarchy, formatting, and visual context.

## The Eight-Stage Conversion Pipeline

### Stage 1: Document Loading and Page Selection

The conversion begins in [`main/pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py) at line 302 within the `to_markdown()` function, which opens the PDF using `pymupdf.open()` and normalizes the target page list at lines 400-404. By default, the pipeline processes all pages, but you can specify selective ranges via the `pages` parameter.

### Stage 2: Header Detection via Font Analysis

PyMuPDF determines document hierarchy by analyzing font sizes to distinguish body text from headings. The `IdentifyHeaders` class (defined at lines 70-84) implements two strategies: scanning every span to identify the most common font size as the body baseline, then mapping larger sizes to corresponding Markdown heading levels (`#` through `######`). The header ID lookup occurs in `IdentifyHeaders.get_header_id()` at lines 53-63. Alternatively, the `TocHeaders` strategy leverages the PDF's existing Table of Contents when available.

### Stage 3: Layout Analysis and Column Detection

Before text extraction, the system analyzes page geometry to handle multi-column layouts and detect tables, images, and vector graphics. The `column_boxes()` helper (imported from `pymupdf4llm.helpers.multi_column`) identifies text blocks, while `refine_boxes()` at lines 221-254 clusters drawings using `page.cluster_drawings` and refines rectangle overlaps to establish reading order.

### Stage 4: Text Extraction and Markdown Formatting

For each text rectangle identified, the `write_text()` function constructs Markdown line-by-line starting at lines 378-382. The implementation retrieves raw spans via `get_raw_lines`, then detects text styling flags (bold, italic, mono, strikethrough) to wrap content with Markdown syntax (`**`, `_`, `` ` ``, `~~`). Hyperlinks are resolved through `resolve_links()` and converted to `[text](url)` format. Heading prefixes are injected via `hdr_string` based on the earlier font size analysis. The span-level formatting logic resides at lines 456-572.

### Stage 5: Table Extraction and Rendering

Tables detected by `page.find_tables` (around lines 188-203) are rendered using `Table.to_markdown()` and inserted before surrounding text blocks via `output_tables()` at lines 404-430. This produces pipe-delimited Markdown tables that maintain tabular structure for LLM parsing.

### Stage 6: Image and Vector Graphics Processing

Visual content is handled based on configuration parameters. The `save_image()` function at lines 665-696 either writes images to disk or embeds them as Base64 data URIs when `embed_images=True`. Vector graphics are managed through the `GRAPHICS_TEXT` constant at line 67, with options to ignore graphics or convert them to image snapshots.

### Stage 7: Final Sanitization

At the conclusion of `write_text()` (lines 782-785), the pipeline performs final cleanup: trimming leading newlines, collapsing duplicate spaces, and replacing null characters with Unicode replacement glyphs to ensure clean Markdown output.

### Stage 8: Output Assembly

The `to_markdown()` function returns the assembled Markdown string at lines 332-335, containing either the complete document or selected pages as a single continuous text block (or as chunks if `page_chunks=True`).

## Why This Format Optimizes LLM Processing

The conversion produces GitHub-flavored Markdown specifically designed for large language model consumption:

- **Explicit document hierarchy**: Heading levels (`#` through `######`) preserve structural semantics that LLMs use for context understanding.
- **Pipe-delimited tables**: Tabular data maintains columnar relationships without complex HTML tags.
- **Inline formatting preservation**: Bold, italic, and code styling provides semantic emphasis markers.
- **Optional image embedding**: Base64-encoded images (`![](data:image/png;base64,...)` ) allow multimodal LLMs to access visual context alongside text.
- **Hyperlink preservation**: External references remain accessible as standard Markdown links.

## Implementation Examples

Basic conversion excluding images:

```python
from main.pymupdf_rag import to_markdown

md_text = to_markdown(
    "reports/annual_report.pdf",
    write_images=False,
    embed_images=False,
    ignore_images=True,
    page_chunks=False,
)

print(md_text)

```

Selective page conversion with image extraction:

```python
md = to_markdown(
    "papers/research.pdf",
    pages=[0, 2, 4],
    write_images=True,
    image_path="./imgs",
    image_format="png",
    table_strategy="lines_strict",
)

```

## Core Files and Architecture

The conversion system resides in the `interviewstreet/hiring-agent` repository with these key components:

- **[`main/pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py)**: Core implementation containing `to_markdown()`, `IdentifyHeaders`, `write_text()`, and image handling logic.
- **[`main/pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/pdf.py)**: Wrapper utilities for file-type validation used by CLI entry points.
- **[`main/transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/transform.py)**: Higher-level CLI interface that orchestrates the conversion pipeline.
- **Dependencies**: Requires `pymupdf` (≥1.25.5) and `pymupdf4llm` as declared in [`requirements.txt`](https://github.com/interviewstreet/hiring-agent/blob/main/requirements.txt) or [`pyproject.toml`](https://github.com/interviewstreet/hiring-agent/blob/main/pyproject.toml).

## Summary

- PyMuPDF converts PDF pages to Markdown for LLM processing through an eight-stage pipeline implemented in [`main/pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py).
- **Header detection** uses font size analysis via `IdentifyHeaders` to map document hierarchy to Markdown heading levels.
- **Layout analysis** handles multi-column documents and identifies tables, images, and graphics before text extraction.
- **Text formatting** preserves inline styling (bold, italic, code) and converts hyperlinks to Markdown syntax during the `write_text()` execution.
- **Table extraction** renders detected tables as pipe-delimited Markdown using `Table.to_markdown()`.
- **Image processing** supports both file output and Base64 embedding via `save_image()` for multimodal LLM contexts.

## Frequently Asked Questions

### What is the primary function for converting PDF to Markdown in PyMuPDF?

The `to_markdown()` function in [`main/pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py) serves as the primary entry point, orchestrating the entire conversion pipeline from document loading through final Markdown assembly.

### How does PyMuPDF determine heading levels during conversion?

The system analyzes font sizes across all text spans using the `IdentifyHeaders` class, establishes the most common size as body text, then maps progressively larger fonts to Markdown heading prefixes (`#` through `######`) via the `get_header_id()` method.

### Can PyMuPDF extract tables from PDFs into Markdown format?

Yes, the pipeline uses `page.find_tables` to detect tabular structures and renders them as pipe-delimited Markdown tables using `Table.to_markdown()`, inserting them at the appropriate positions within the text flow.

### Does the conversion support image extraction for multimodal LLMs?

The `save_image()` function handles images by either writing them to disk or embedding them as Base64 data URIs directly in the Markdown, allowing multimodal LLMs to process visual content alongside text when `embed_images=True` is specified.