How pdf-inspector Supports Tagged PDFs and Structure Tree Extraction

pdf-inspector detects /StructTreeRoot entries in PDF documents to parse semantic structure trees, converting tags like <H1>, <P>, and <Code> directly into Markdown elements while maintaining logical document hierarchy.

The firecrawl/pdf-inspector library is designed to preserve the logical structure of tagged PDFs by extracting their embedded structure trees and converting them into semantically rich Markdown. When a PDF contains a /StructTreeRoot object, the tool reads the hierarchy directly rather than inferring structure from visual clues alone, producing more accurate output for academic papers, legal contracts, and technical manuals.

Detecting the Structure Tree

During the extraction phase, pdf-inspector checks for the presence of a /StructTreeRoot entry in the PDF catalog. This detection occurs in src/extractor/mod.rs, where the parser determines whether the document contains a formal structure tree or requires heuristic analysis.

If a structure tree is present, the library traverses the tree nodes to extract the semantic role of each content element. This traversal logic is implemented in src/structure_tree.rs, which builds an internal representation of the document's hierarchy before conversion begins.

Mapping PDF Roles to Markdown Elements

The core conversion logic maps standard PDF structure roles to their Markdown equivalents. This mapping is defined in src/markdown/classify.rs and utilized during the conversion stage in src/markdown/convert.rs.

The library handles the following role transformations:

• H1 through H6 → Markdown headings (# through ######)
• P → Paragraph breaks (\n\n)
• L → List items (- )
• Code → Fenced code blocks (```)
• BlockQuote → Block-quote lines (> )

Each role is stored internally as a MarkdownRole enum variant, ensuring type-safe conversion throughout the pipeline.

Extracting the Structure Tree

The src/structure_tree.rs module parses the /StructTreeRoot dictionary and recursively processes child elements to reconstruct the document's logical reading order. This module handles the complexity of PDF structure trees, including parent-child relationships between content items and their associated standard structure types.

By resolving these relationships, pdf-inspector can reconstruct nested lists, preserve heading hierarchies, and distinguish between code blocks and regular paragraphs based on the author's original intent rather than font characteristics.

Fallback Heuristics for Untagged PDFs

When a PDF lacks a /StructTreeRoot entry, pdf-inspector falls back to font-size and style heuristics implemented in markdown::analysis.rs. This secondary analysis examines text attributes such as bolding, font size changes, and indentation patterns to infer headings, lists, and block quotations.

While this heuristic approach is effective for many documents, it cannot match the precision of explicit tagging for complex layouts or documents with non-standard styling conventions.

Code Examples

Rust API Usage

Use the process_pdf_with_options function from src/lib.rs to extract a tagged PDF with default structure tree preservation:

use pdf_inspector::process_pdf_with_options;
use pdf_inspector::ProcessOptions;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Enable structure‑tree extraction (default)
    let opts = ProcessOptions::default();

    // Path to a tagged PDF
    let md = process_pdf_with_options("examples/tagged.pdf", opts)?;
    println!("{}", md);
    Ok(())
}

Command Line Interface

The pdf2md CLI tool automatically detects and preserves structure trees when converting PDFs:

pdf2md examples/tagged.pdf > output.md

# The generated Markdown retains headings, lists, code blocks, etc.

Python Bindings

Access the parsed structure tree directly through the Python API to inspect the document hierarchy:

import pdf_inspector

doc = pdf_inspector.process_pdf("examples/tagged.pdf")
print(doc.structure_tree)   # Shows the hierarchy of PDF roles

Summary

• Structure detection occurs in src/extractor/mod.rs, which identifies /StructTreeRoot entries during the initial PDF parsing phase.
• Tree traversal is handled by src/structure_tree.rs, building a complete hierarchy of semantic roles from the PDF's structure elements.
• Role mapping in src/markdown/classify.rs converts PDF standard structure types into MarkdownRole enum variants for type-safe processing.
• Markdown generation happens in src/markdown/convert.rs, emitting token-efficient output that preserves the original document's logical structure.
• Fallback analysis in markdown::analysis.rs provides heuristic-based structure inference for PDFs without embedded tags.

Frequently Asked Questions

What is a tagged PDF?

A tagged PDF is a document that includes a structure tree (defined via the /StructTreeRoot dictionary) containing semantic markup such as <H1>, <P>, or <Code> tags. Unlike untagged PDFs that only describe visual appearance, tagged PDFs encode the logical reading order and document hierarchy, enabling assistive technologies and parsers like pdf-inspector to understand content semantics.

How does pdf-inspector handle PDFs without structure trees?

For untagged PDFs, pdf-inspector falls back to heuristic analysis implemented in markdown::analysis.rs. The library examines font sizes, text styles, and indentation patterns to infer headings, lists, and block types. While this approach works for many documents, it is less reliable than explicit structure trees for complex layouts or documents with ambiguous visual formatting.

What Markdown elements are supported for tagged PDF conversion?

pdf-inspector converts standard PDF structure roles into corresponding Markdown elements: headings (H1-H6), paragraphs, unordered lists, fenced code blocks, and block quotes. The mapping occurs through the MarkdownRole enum, ensuring that semantic intent from the PDF is preserved in the final Markdown output.

Can I access the raw structure tree programmatically?

Yes. When using the Python bindings, the processed document object exposes a structure_tree attribute containing the full hierarchy of parsed PDF roles. In Rust, you can inspect the intermediate representation before Markdown conversion by examining the tree built in src/structure_tree.rs prior to the conversion stage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}