How to Programmatically Use pdf-inspector in Rust: Complete API Guide

Use pdf-inspector from Rust by calling process_pdf() for text-based PDFs or process_pdf_with_ocr() for scanned documents, both returning a PdfResult with Markdown output and detection metadata.

pdf-inspector is a pure-Rust library for fast PDF classification, text extraction, layout analysis, and Markdown conversion. According to the firecrawl/pdf-inspector source code, the library follows a three-stage pipeline—Detection, Extraction, and Markdown Generation—with a deliberately single-load design that eliminates redundant I/O by sharing internal representations between stages.

Setting Up pdf-inspector in Your Rust Project

Add the crate to your Cargo.toml:

cargo add pdf-inspector

Or manually add:

[dependencies]
pdf-inspector = "1"

For OCR support, enable the ocr feature:

[dependencies]
pdf-inspector = { version = "1", features = ["ocr"] }

Core API: Processing PDFs Programmatically

All public functions are defined in src/lib.rs and re-exported through the crate root. The two primary entry points are:

Function	Purpose	OCR Required
process_pdf()	Fast extraction for text-based PDFs	No
process_pdf_with_ocr()	Selective OCR for scanned or mixed-content PDFs	Yes (ocr feature)

Basic Extraction Without OCR

For fully text-based PDFs, use the simplest API:

use pdf_inspector::process_pdf;

fn main() -> anyhow::Result<()> {
    // Load from file path
    let result = process_pdf("example.pdf")?;

    // Access detection metadata
    println!("PDF type: {:?}", result.pdf_type);

    // Retrieve generated Markdown
    if let Some(md) = result.markdown {
        println!("--- Markdown ---\n{md}");
    }

    // Check which pages lack extractable text
    if !result.pages_needing_ocr.is_empty() {
        println!("Pages needing OCR: {:?}", result.pages_needing_ocr);
    }

    Ok(())
}

The PdfResult struct returned by process_pdf contains:

• pdf_type: Classification as TextBased, Scanned, ImageBased, or Mixed
• markdown: Optional String with clean Markdown output
• pages_needing_ocr: Vector of page numbers requiring OCR processing

Selective OCR for Scanned Documents

When the ocr feature is enabled, process_pdf_with_ocr() runs OCR only on pages that lack extractable text—avoiding the overhead of processing already-readable content:

use pdf_inspector::process_pdf_with_ocr;

fn main() -> anyhow::Result<()> {
    let result = process_pdf_with_ocr("scanned_mix.pdf")?;

    println!("PDF type: {:?}", result.pdf_type);
    
    if let Some(md) = result.markdown {
        println!("Markdown with OCR:\n{md}");
    }
    
    // Pages where OCR was actually applied
    println!("OCR applied to: {:?}", result.pages_routed_to_ocr);
    
    Ok(())
}

The OCR runtime (PDFium + ONNX) is loaded lazily and invoked only for pages routed through pages_needing_ocr, as implemented in src/lib.rs.

Customizing Extraction Behavior

Use PdfOptions and the builder pattern to tune detection strategies, enable page selection, and control performance characteristics:

use pdf_inspector::{PdfOptions, ScanStrategy};

fn main() -> anyhow::Result<()> {
    // Sample only 5 pages for large PDFs (faster detection)
    let opts = PdfOptions::builder()
        .scan_strategy(ScanStrategy::Sample(5))
        .build();

    let result = pdf_inspector::process_pdf_with_options("large.pdf", opts)?;
    println!("Detected type: {:?}", result.pdf_type);
    
    Ok(())
}

PdfOptions and ScanStrategy are defined in src/lib.rs alongside the core API functions.

Understanding the Three-Stage Pipeline

To use pdf-inspector effectively in Rust, it helps to understand how the library processes documents internally:

Stage 1: Detection (src/detector.rs)

The detector quickly classifies PDFs by sampling content streams and analyzing operators:

• Counts Tj/TJ text operators vs. Do image operators
• Returns a confidence score and classification (TextBased, Scanned, ImageBased, or Mixed)
• Determines which pages require OCR routing

Stage 2: Extraction (src/extractor/mod.rs)

The extractor performs a single complete parse of the PDF:

• Walks each page's content stream
• Collects TextItem structs with Unicode text, font metadata, and X/Y coordinates
• Resolves fonts and ToUnicode CMaps via src/extractor/fonts.rs
• Builds layout graphs for column detection in src/extractor/layout.rs
• Detects tables using three strategies: rectangle-based, heuristic alignment, then line-grid

Stage 3: Markdown Generation (src/markdown/convert.rs)

The converter transforms extracted items through a structured pipeline:

• Analyzes font sizes to infer heading levels
• Detects lists, code blocks, tables, and URLs
• Applies post-processing: hyphenation fixing, drop-cap merging, page-break insertion
• Emits token-efficient Markdown

Loading PDFs from Memory

For applications processing in-memory buffers rather than files:

use pdf_inspector::load_document_from_mem;

fn process_bytes(pdf_data: &[u8]) -> anyhow::Result<()> {
    // Same internal representation as file-based loading
    let result = pdf_inspector::process_pdf_from_mem(pdf_data)?;
    // ... use result
    Ok(())
}

The single-load design ensures load_document_from_path and load_document_from_mem share the same optimized internal structures.

Key Source Files for Advanced Usage

File	Responsibility	When to Reference
src/lib.rs	Public API, PdfOptions builder	All integration work
src/detector.rs	PDF-type detection logic	Custom classification needs
src/extractor/mod.rs	Core extraction orchestration	Understanding text extraction
src/extractor/fonts.rs	Font metrics, CMap resolution	Debugging encoding issues
src/extractor/layout.rs	Column detection, reading order	Multi-column document handling
src/tables/detect_rects.rs	Rectangle-based table detection	Table extraction debugging
src/markdown/convert.rs	Markdown generation pipeline	Output customization
src/bin/pdf2md.rs	CLI reference implementation	Debugging and testing patterns

Summary

• process_pdf() is the fastest path for text-based PDFs—no feature flags required
• Enable the ocr feature and use process_pdf_with_ocr() for mixed or scanned documents with selective page processing
• Configure PdfOptions to control sampling strategies and performance for large files
• Load from paths or memory using the unified single-load API in src/lib.rs
• The three-stage pipeline—Detection, Extraction, Markdown Generation—shares internal state to eliminate redundant I/O

Frequently Asked Questions

How do I enable OCR support in pdf-inspector?

Add the ocr feature to your Cargo.toml dependency: pdf-inspector = { version = "1", features = ["ocr"] }. Then call process_pdf_with_ocr() instead of process_pdf(). The OCR runtime loads only for pages lacking extractable text.

Can I process PDFs from memory buffers instead of files?

Yes. Use load_document_from_mem() or process_pdf_from_mem() to process &[u8] buffers. These functions share the same internal representation as file-based loading, maintaining the library's single-load efficiency.

What PDF types can pdf-inspector detect?

The detector in src/detector.rs classifies PDFs as TextBased (fully extractable text), Scanned (images only), ImageBased (image-heavy with minimal text), or Mixed (combined content). Detection samples content streams and returns confidence scores without full parsing.

How does selective OCR work?

When process_pdf_with_ocr() is called, the library first runs detection to identify pages_needing_ocr. The OCR runtime is only invoked for those specific pages, leaving text-based pages to the fast extraction path. This minimizes processing time and resource usage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}