# How to Extract Semantic Structure from Tagged PDFs Using pdf‑inspector

> Extract semantic structure from tagged PDFs with pdf-inspector. Map headings, paragraphs, tables, and figures to text content. Convert PDFs to structured Markdown or JSON easily.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: how-to-guide
- Published: 2026-08-13

---

**pdf‑inspector reads the `/StructTreeRoot` hierarchy of tagged PDFs to map every semantic element—headings, paragraphs, tables, and figures—to its corresponding text content via Marked Content IDs (MCIDs), enabling conversion to structured Markdown or JSON.**

The `firecrawl/pdf‑inspector` Rust library provides a robust pipeline for parsing PDF tagging information according to ISO standards. Unlike simple text extraction tools that lose document hierarchy, pdf‑inspector preserves semantic roles by joining the PDF’s structure tree with raw content streams, allowing you to convert tagged PDFs into Markdown that respects original heading levels, list structures, and table semantics.

## Understanding Tagged PDFs and the Structure Tree

Tagged PDFs embed a **structure tree** (located at `/StructTreeRoot` in the PDF catalog) that defines logical document elements independently from visual layout. Each node in this tree represents a semantic component—such as headings (`H1`–`H6`), paragraphs (`P`), tables (`Table`), or table cells (`TD`)—and references specific content through **Marked Content IDs (MCIDs)**.

When pdf‑inspector processes a document, it resolves these references against the page’s content streams to produce a mapping between text content and its structural role. This approach allows the library to handle custom role mappings defined in the PDF’s `/RoleMap` dictionary while gracefully falling back to untagged extraction when no structure tree exists.

## The Three-Step Extraction Pipeline

The extraction process implemented in `firecrawl/pdf‑inspector` follows a distinct three-phase architecture to separate parsing from content retrieval.

### Parse the Structure Tree

The entry point `StructTree::from_doc` in [`src/structure_tree.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/structure_tree.rs) (line 301) traverses the `/StructTreeRoot` dictionary to build an in-memory tree of `StructElement` nodes. Each node stores:

- A `StructRole` (standard or custom-mapped)
- Optional alternate text and language attributes
- A list of `MarkedContentRef` leaf references pointing to specific content instances

If the PDF lacks tagging information, this function returns `None`, allowing callers to proceed with unstructured text extraction.

### Build the MCID-to-Role Lookup

Once the tree is parsed, `StructTree::mcid_to_roles` (line 345 in [`src/structure_tree.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/structure_tree.rs)) inverts the hierarchy into a per-page `HashMap<MCID, StructRole>`. This lookup table enables O(1) resolution of semantic roles during text extraction, mapping each integer MCID to its corresponding structural label such as `"H1"`, `"P"`, or `"Table"`.

### Join with Text Content

The public API function `extract_structure_elements_mem` (defined in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs), line 93) orchestrates the final assembly. It loads the PDF, obtains the structure tree, builds the MCID-to-role map, and returns a flat vector of `StructureElement` records. These records contain:

- 1-indexed page numbers
- MCID integers
- Role name strings (e.g., `"H1"`, `"TD"`)

Consumers then join these elements with `TextItem` instances from `extract_text_with_positions` (located in [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs)), which also carry MCID and page metadata, producing fully-qualified semantic output.

## Practical Implementation

### Basic Rust Example

The following snippet demonstrates loading a tagged PDF, extracting both structural metadata and text positions, and joining them to produce semantically labeled output:

```rust
use pdf_inspector::{
    extract_text_with_positions,
    extract_structure_elements,
};

fn main() -> Result<(), pdf_inspector::PdfError> {
    // Extract raw text items including MCIDs and page numbers
    let text_items = extract_text_with_positions("document.pdf")?;
    
    // Extract semantic roles from the structure tree
    let struct_elems = extract_structure_elements("document.pdf", None)?;
    
    // Build lookup table: (page, mcid) → role
    let mut role_lookup = std::collections::HashMap::new();
    for elem in struct_elems {
        role_lookup.insert((elem.page, elem.mcid), elem.role);
    }
    
    // Attach semantic roles to extracted text
    for item in text_items {
        match role_lookup.get(&(item.page, item.mcid)) {
            Some(role) => println!("[{}] {}", role, item.text),
            None => println!("[P] {}", item.text), // Fallback paragraph
        }
    }
    
    Ok(())
}

```

### Command-Line Usage

The shipped `pdf2md` binary provides immediate access to structure extraction without writing code:

```bash
pdf2md --json --extract-structure input.pdf

```

This outputs a JSON array of structural elements:

```json
[
  {"page":1,"mcid":12,"role":"H1"},
  {"page":1,"mcid":34,"role":"P"},
  {"page":2,"mcid":7,"role":"Table"}
]

```

When combined with the standard `--json` text output, this data allows you to reconstruct headings with proper Markdown depth (`#`, `##`) rather than inferring hierarchy from font sizes.

## Key Source Files and Architecture

| File | Purpose | Key Components |
|------|---------|----------------|
| [`src/structure_tree.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/structure_tree.rs) | Parses `/StructTreeRoot` and manages role mapping | `StructTree::from_doc`, `StructTree::mcid_to_roles`, `StructElement`, `StructRole` |
| [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) | Public API façade | `extract_structure_elements_mem`, `extract_structure_elements`, `StructureElement` |
| [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) | Extracts text with geometric and MCID metadata | `extract_text_with_positions`, `TextItem` |
| [`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs) | Converts extracted items to Markdown while respecting roles | Role-aware conversion pipeline |
| [`src/types.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/types.rs) | Core type definitions | `TextItem`, `PdfRect`, `MarkedContentRef` |

## Summary

- **pdf‑inspector** extracts semantic structure by parsing the `/StructTreeRoot` dictionary found in tagged PDFs.
- The pipeline uses `StructTree::from_doc` to build the hierarchy, `StructTree::mcid_to_roles` to create fast lookup tables, and `extract_structure_elements_mem` to expose the data.
- Text items extracted via `extract_text_with_positions` carry MCID metadata that can be joined with structural roles to preserve headings, tables, and lists during Markdown conversion.
- The architecture gracefully handles untagged PDFs by returning empty vectors, allowing fallback to geometric-based extraction.

## Frequently Asked Questions

### What is the difference between tagged and untagged PDF extraction?

Tagged PDFs contain explicit semantic markup that identifies headings, paragraphs, and other logical elements regardless of visual formatting. pdf‑inspector extracts this via the structure tree to preserve document meaning. Untagged PDFs lack this metadata, forcing the library to rely solely on geometric heuristics for structure inference.

### Can pdf‑inspector handle custom role mappings?

Yes. When a PDF defines a `/RoleMap` dictionary that maps custom role names to standard PDF structure types, `StructTree::from_doc` in [`src/structure_tree.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/structure_tree.rs) resolves these mappings during tree construction. This ensures that proprietary tags are normalized to standard roles like `"H1"` or `"Table"` before reaching the output pipeline.

### How do I combine structural elements with the actual text content?

Call `extract_text_with_positions` to obtain a vector of `TextItem` objects, each containing `page`, `mcid`, and `text` fields. Then call `extract_structure_elements` to get the corresponding role mappings. Join these datasets using the tuple `(page, mcid)` as the key, as demonstrated in the Rust example above.

### What happens if I run structure extraction on an untagged PDF?

The `extract_structure_elements` function will return an empty vector. Because pdf‑inspector separates tree parsing from content extraction in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs), untagged documents simply skip the semantic enhancement phase while still allowing standard text extraction to proceed normally.