# PDF to Markdown Conversion Behavior in pdf-inspector: Default Pipeline Explained

> Discover the default PDF to Markdown conversion in pdf-inspector. Learn how the Rust pipeline transforms PDFs into structured Markdown quickly and efficiently.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-08-13

---

**The default `pdf2md` conversion in firecrawl/pdf-inspector executes a deterministic, fully-Rust pipeline that transforms PDFs into structured Markdown without OCR, completing in approximately 150 milliseconds for text-based documents by detecting document types, extracting positioned text lines, and applying layout-aware formatting rules.**

The firecrawl/pdf-inspector repository provides a high-performance Rust library and CLI tool for converting PDF documents to Markdown. Understanding the default PDF to Markdown conversion behavior is essential for developers integrating this tool into AI preprocessing pipelines or document processing workflows. The default configuration prioritizes speed and structural accuracy over visual fidelity, bypassing OCR entirely for text-based documents while preserving logical document hierarchy through font analysis and geometric detection.

## The Three-Stage Default Conversion Pipeline

The default conversion follows a deterministic three-stage architecture implemented across the [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs), `src/extractor/`, and `src/markdown/` modules. This pipeline processes the PDF sequentially without redundant I/O operations.

### Stage 1: Document Type Detection

Before extraction begins, [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) performs a rapid scan of the PDF's content streams to classify the document as **TextBased**, **Scanned**, **ImageBased**, or **Mixed**. This classification completes in approximately 10-50 milliseconds and produces a confidence score alongside a per-page list identifying which pages require OCR processing. The default conversion path skips OCR entirely when the document is classified as TextBased, enabling the sub-150ms processing times characteristic of the standard pipeline.

### Stage 2: Content Extraction

The system loads the PDF **once** and walks every page's content stream via the modules in `src/extractor/`. Key components include:

- **[`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs)**: Parses raw PDF content streams
- **[`src/extractor/fonts.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/fonts.rs)**: Resolves font dictionaries, CMap tables, and ToUnicode mappings
- **[`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs)**: Determines column structures and reading order

This stage gathers **TextLine** objects containing X/Y coordinates, font sizes, and style flags (bold, italic, underline). It simultaneously extracts table geometry through drawing operation analysis, identifies embedded images, and collects hyperlink annotations.

### Stage 3: Markdown Generation

Finally, [`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs) transforms the extracted **TextLine** objects into Markdown syntax. The implementation applies deterministic rules to reconstruct document structure without preserving visual padding that would waste token space in downstream AI applications.

## Default Markdown Formatting Rules

The [`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs) module applies specific heuristics to convert raw text positions into semantic Markdown elements:

- **Headings**: The system detects H1-H4 headings by analyzing font-size tiers relative to the most common body text size. Larger font sizes trigger heading classification based on relative thresholds.

- **Lists**: Bullet lists (recognizing `•`, `-`, `*`, and other markers) and numbered/lettered lists are identified through leading character analysis and indentation patterns.

- **Code Blocks**: Monospace font detection or explicit PDF `StructRole::Code` tags trigger fenced code block formatting in the output.

- **Tables**: The converter first attempts rectangle-based detection using union-find algorithms on drawing operations, falling back to heuristic text-alignment detection when geometric boundaries are ambiguous.

- **Text Styling**: Bold, italic, and underline formatting are applied when font names or ToUnicode mappings signal these styles in the source PDF.

- **Hyperlinks**: URL annotations are converted to standard Markdown link syntax `[text](url)`.

- **Page Breaks**: The default configuration inserts HTML comments `<!-- Page N -->` at page boundaries to delineate pagination without disrupting Markdown rendering.

- **Post-Processing**: The pipeline collapses dot-leaders (table of contents lines), removes spurious hyphenation artifacts, strips page numbers from headers/footers, merges drop-cap characters with their subsequent text, and normalizes excessive whitespace.

## Using the Default Conversion

The default behavior activates automatically when running `pdf2md` without optional flags. All code examples below demonstrate the standard, non-OCR conversion path.

### Command Line Interface

Install the binary and convert documents with zero configuration:

```bash

# Install from crates.io or build from source

cargo install pdf-inspector

# Default conversion (no OCR, standard formatting)

pdf2md report.pdf

```

The command outputs Markdown with headings, lists, tables, and page-break comments as described above.

### Rust Library API

Integrate the default pipeline programmatically using the `process_pdf` function from [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs):

```rust
use pdf_inspector::process_pdf;

fn main() -> Result<(), pdf_inspector::Error> {
    let result = process_pdf("report.pdf")?;
    
    println!("PDF type: {:?}", result.pdf_type);  // e.g., TextBased
    if let Some(md) = result.markdown {
        println!("{}", md);  // Default Markdown output
    }
    Ok(())
}

```

The `process_pdf` function automatically applies default `PdfOptions` and `MarkdownOptions`, triggering the three-stage pipeline without OCR.

### Python and Node.js Bindings

Language bindings maintain identical default behavior:

```python
import pdf_inspector

result = pdf_inspector.process_pdf("report.pdf")
print(result.pdf_type)      # "text_based", "scanned", etc.

print(result.markdown)      # Default Markdown string

```

```javascript
import { readFileSync } from "fs";
import { processPdf } from "@firecrawl/pdf-inspector";

const pdf = readFileSync("report.pdf");
const result = processPdf(pdf);
console.log(result.pdfType);   // "TextBased", "Scanned", etc.
console.log(result.markdown);  // Default Markdown output

```

## Key Source Files and Implementation Details

The default conversion behavior is distributed across specific modules:

- **[`src/bin/pdf2md.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/bin/pdf2md.rs)**: CLI entry point that parses arguments and invokes `process_pdf` with default `MarkdownOptions`

- **[`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs)**: Public API exposing `process_pdf` and `PdfOptions` builders that glue detection, extraction, and conversion stages together

- **[`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs)**: Core line-to-Markdown loop implementing heading, list, table, and code block detection heuristics

- **[`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)**: Fast PDF-type classifier determining OCR necessity before extraction begins

- **[`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs)**, **[`src/extractor/fonts.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/fonts.rs)**, **[`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs)**: Extract positioned **TextLine** objects, resolve font mappings, and determine reading order

## Summary

- **The default pipeline runs without OCR**, completing in ~150ms for text-based PDFs by leveraging Rust's performance and single-pass extraction
- **Three stages drive conversion**: Document type detection ([`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)), content extraction (`src/extractor/`), and Markdown generation ([`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs))
- **Structure is inferred from fonts and geometry**: Headings derive from font-size tiers, lists from leading markers, tables from drawing operations or alignment heuristics
- **Clean output is prioritized**: The default settings collapse visual padding, remove headers/footers, and insert page-break comments rather than attempting pixel-perfect reproduction
- **Multiple interfaces share behavior**: The CLI `pdf2md`, Rust `process_pdf`, Python, and Node.js APIs all execute the same default pipeline when customization options are omitted

## Frequently Asked Questions

### Does the default conversion use OCR?

No. By default, pdf-inspector skips OCR entirely for documents classified as **TextBased** during the detection stage in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs). The system only flags pages for OCR processing if the detector classifies them as **Scanned**, **ImageBased**, or **Mixed**, and even then, OCR requires explicit opt-in via command-line flags or API options. This design choice ensures sub-150ms conversion times for standard text-based PDFs.

### How does pdf-inspector detect headings and lists?

The system analyzes **TextLine** objects extracted in [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) to identify document structure. **Headings** are detected through font-size tier analysis in [`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs), comparing each line's font size against the most common body text size to assign H1-H4 levels. **Lists** are recognized by scanning for bullet markers (`•`, `-`, `*`) or numbered/lettered prefixes at the beginning of text lines, then grouping items by indentation levels.

### What file handles the core Markdown transformation?

The primary logic resides in **[`src/markdown/convert.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs)**. This module implements the line-to-Markdown conversion loop, applying default rules for headings, lists, code blocks, tables, and inline styling. It receives structured **TextLine** objects from the extraction stage and outputs clean Markdown, handling post-processing steps like hyphenation removal and whitespace normalization before returning the final string.

### Can I customize the default Markdown output?

Yes, though the defaults are optimized for token efficiency. The `process_pdf` function in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) accepts optional configuration structs that modify behavior. Command-line users can pass flags like `--compact` to reduce whitespace further or `--pages` to limit conversion to specific page ranges. However, omitting these flags executes the standard pipeline described above, which preserves structural elements while minimizing visual noise.