How pdf-inspector Uses Tagged PDF Structure for Semantic Classification: A Technical Deep Dive

pdf-inspector extracts the structure tree embedded in tagged PDFs to map semantic roles such as H1-H6, P, L, and Code directly into Markdown, preserving author intent with higher fidelity than visual heuristics alone.

The firecrawl/pdf-inspector project leverages PDF accessibility features to convert documents into semantically rich Markdown. When a PDF contains a structure tree—the hierarchical metadata that defines headings, paragraphs, lists, and other elements—pdf-inspector prioritizes these tags over layout-based guessing. This approach ensures that a heading marked as H1 in the PDF becomes a proper # heading in Markdown, even if visual styling is misleading.

Parsing the PDF Structure Tree

The foundation of semantic classification begins in src/structure_tree.rs. This module walks the PDF's /StructTreeRoot dictionary, the root of the document's accessibility hierarchy.

The parser performs three critical operations:

1. Navigates the structure hierarchy – Recursively descends through /K (kids) arrays to discover all marked content
2. Extracts role information – Maps each node's /S (standard) entry or /Alt (alternative) entry to an internal enum
3. Builds StructureRole enums – Converts PDF tag names like /H1, /P, /L, /Code into typed variants for downstream consumption

The /StructTreeRoot dictionary is defined in the PDF ISO 32000 standard and is required for PDF/UA (universal accessibility) compliance. pdf-inspector treats this as the authoritative source of document semantics when present.

Integrating Tags with Markdown Classification

Once structure roles are extracted, src/markdown/classify.rs performs the semantic labeling. This module receives sequences of text items alongside their associated StructureRole values.

The classification logic follows a strict precedence:

• Tagged roles take priority – If a StructureRole exists for a content region, the classifier applies the corresponding semantic label: Header(1), Header(2), Paragraph, ListItem, CodeBlock, Figure, etc.
• Heuristics serve as fallback – When tagging is absent, pdf-inspector infers structure from font size, weight, indentation, and spatial positioning
• Role conflicts resolve to tags – If visual heuristics contradict explicit tags, the tagged role wins

This design principle ensures that documents created with accessibility in mind—such as those exported from Microsoft Word with "Create PDF/A" enabled—retain their intended semantics through conversion.

Post-Processing with Semantic Labels

The semantic labels drive refinement in src/markdown/postprocess.rs. With accurate role information, this module performs targeted cleanup:

Label Applied	Post-Processing Action
Header(n)	Merges split heading lines, normalizes levels
ListItem	Groups items into proper list blocks, handles nesting
CodeBlock	Preserves whitespace boundaries, detects language hints
Paragraph	Handles drop caps, widow/orphan prevention

Because these operations know the intended role rather than guessing from pixels, the resulting Markdown requires less manual correction.

Practical Implementation

Rust API Usage

Process a tagged PDF with structure preservation enabled:

use pdf_inspector::process_pdf_with_options;
use pdf_inspector::ProcessOptions;

fn main() {
    let opts = ProcessOptions {
        keep_structure_tree: true,
        ..Default::default()
    };

    let markdown = process_pdf_with_options("sample_tagged.pdf", opts)
        .expect("failed to process PDF");

    println!("{}", markdown);
}

The keep_structure_tree: true flag ensures the pipeline prioritizes /StructTreeRoot data over pure visual analysis.

CLI Execution

The command-line tool exposes identical functionality:

pdf2md --json sample_tagged.pdf

Example output showing direct tag-to-Markdown mapping:

{
  "content": "# Title\n\nParagraph text.\n\n- Item 1\n- Item 2\n\n```rust\nfn main() {}\n```"

}

Here, # Title originates from an H1 tag, list items from L tags, and the fenced code block from a Code tag—all extracted from the PDF's accessibility structure.

Key Source Files

Understanding the implementation requires familiarity with these modules:

• src/structure_tree.rs – Parses /StructTreeRoot, converts PDF dictionary structures into StructureRole enums
• src/markdown/classify.rs – Assigns semantic labels using tagged roles when available
• src/markdown/postprocess.rs – Applies label-aware transformations for final Markdown output
• src/lib.rs – Public API entry point (process_pdf_with_options) that orchestrates the pipeline

Summary

• pdf-inspector prioritizes tagged PDF structure over visual heuristics when /StructTreeRoot is present
• src/structure_tree.rs extracts and normalizes semantic roles from PDF accessibility dictionaries
• src/markdown/classify.rs maps extracted roles to Markdown constructs like headers, lists, and code blocks
• Semantic labels enable intelligent post-processing in src/markdown/postprocess.rs for cleaner output
• The keep_structure_tree option controls whether to honor PDF tags or rely on layout inference

Frequently Asked Questions

What is a tagged PDF structure tree?

A tagged PDF contains a /StructTreeRoot dictionary that defines a semantic hierarchy parallel to the visual content. This structure identifies headings, paragraphs, lists, tables, and figures with standardized role tags. pdf-inspector consumes this tree to produce Markdown that matches the document's logical organization rather than just its visual appearance.

How does pdf-inspector handle PDFs without tags?

When no /StructTreeRoot exists, pdf-inspector falls back to heuristic analysis based on font properties, positioning, and spatial grouping. These heuristics estimate semantic roles but are less reliable than explicit tags—especially for documents with unconventional layouts or styling.

Can I disable structure tree processing?

Yes. Omit the keep_structure_tree option or set it to false in ProcessOptions. This forces pure visual analysis, which may be desirable for scanned documents or PDFs with corrupted tag structures where heuristics outperform broken accessibility data.

Which PDF generators produce good structure trees?

Modern word processors and design tools with accessibility features generate reliable tagging: Microsoft Word (PDF/A export), Adobe InDesign (tagged PDF option), LaTeX with pdfx package, and Google Docs. Scanned documents and legacy generators typically lack meaningful structure trees.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}