How to Use pdf-inspector for Smart PDF Routing to Avoid OCR Costs

Use pdf-inspector's classify_pdf_mem and extract_pages_markdown_mem functions to detect which pages need OCR before running any expensive inference, then route only flagged pages to your OCR engine.

The pdf-inspector Rust library from Firecrawl implements a deterministic pre-flight analysis system that examines PDF internal structure to minimize unnecessary OCR compute. By parsing the document once and analyzing page-level signals—scanned raster images, vector-drawn text, broken font encodings, and GID-encoded fonts—the library produces per-page routing decisions that let you extract clean text directly while isolating only problematic pages for OCR fallback.

---

The Core Smart Routing API

Two public functions in src/lib.rs form the complete routing interface:

• classify_pdf_mem – Fast classification returning PDF type, page count, and pages_needing_ocr list
• extract_pages_markdown_mem – Full extraction with per-page Markdown and OCR flags

Both operate on in-memory buffers, making them suitable for serverless pipelines and streaming architectures.

---

How the Routing Pipeline Works

Step 1: Single-Pass PDF Loading

The document is parsed once using a shared internal representation. This shared structure feeds both detection and extraction phases, eliminating redundant I/O and parsing overhead.

Step 2: OCR-Need Detection

The detect_from_document function in src/detector.rs analyzes each page for:

• Page-level raster images – indicates scanned documents requiring OCR
• Vector-drawn text – pure vector pages can bypass OCR entirely
• Broken font encodings – CID garbage and encoding issues that produce unreadable text
• GID-encoded fonts – glyphs without proper Unicode mappings

The detector populates pages_needing_ocr with 0-based page indices and provides machine-readable ocr_reason values for each flagged page.

Step 3: Direct Text Extraction

Pages not flagged for OCR are processed by the content-stream parser in src/extractor/content_stream.rs. This builds TextItem objects containing raw text, font size, and positional coordinates—no OCR required.

Step 4: Layout Analysis

Cheap in-memory analysis in src/markdown/analysis.rs determines table structures, column layouts, and font-size hierarchies. This informs the markdown converter whether a page qualifies as "complex" or simple text.

Step 5: Per-Page Markdown Generation

The extract_pages_markdown_mem function produces PageMarkdown structs with three fields:

• markdown – rendered Markdown (empty string when needs_ocr is true)
• needs_ocr – boolean routing flag
• ocr_reason – human-readable explanation (e.g., scanned, suspected_garbled_text, vector_text)

Step 6: Hybrid OCR Fallback

Your application code receives the needs_ocr flags and routes only those specific pages to GPU-accelerated OCR engines like Tesseract or EasyOCR. Deterministic routing eliminates network calls during detection, making it safe for untrusted PDFs.

---

Complete Code Examples

Quick Classification (Memory Buffer)

Use classify_pdf_mem for the fastest possible routing decision when you only need to know which pages require OCR:

use pdf_inspector::{classify_pdf_mem, PdfError};

fn route_pdf(buf: &[u8]) -> Result<(), PdfError> {
    // Fast classification – tells us which pages need OCR.
    let classification = classify_pdf_mem(buf)?;
    println!("PDF type: {:?}", classification.pdf_type);
    println!("Total pages: {}", classification.page_count);
    println!("Pages needing OCR (0-based): {:?}", classification.pages_needing_ocr);
    Ok(())
}

Source: src/lib.rs lines 3890-3904

Per-Page Extraction with Routing Flags

Use extract_pages_markdown_mem when you need both routing decisions and extracted Markdown:

use pdf_inspector::{extract_pages_markdown_mem, PdfError};

fn extract_and_route(buf: &[u8]) -> Result<(), PdfError> {
    // Get per-page Markdown plus OCR flags.
    let result = extract_pages_markdown_mem(buf, None)?; // `None` → all pages

    for page in result.pages {
        if page.needs_ocr {
            // Send this page to an OCR service.
            println!("Page {} needs OCR (reason: {:?})", page.page + 1, page.ocr_reason);
        } else {
            // Use the extracted Markdown directly.
            println!("Page {} markdown:\n{}", page.page + 1, page.markdown);
        }
    }
    Ok(())
}

Source: src/lib.rs lines 4380-4387

Region-Based Extraction for Layout Pipelines

For advanced workflows that integrate with layout models or need selective extraction:

use pdf_inspector::{extract_text_in_regions_mem, PdfError};

fn extract_regions(buf: &[u8]) -> Result<(), PdfError> {
    // Define regions as (page_0_idx, Vec<[x1, y1, x2, y2]>)
    let regions = vec![
        (0, vec![[100.0, 200.0, 300.0, 250.0]]), // first page, one bbox
        (2, vec![[50.0, 400.0, 550.0, 450.0]])  // third page, another bbox
    ];

    let results = extract_text_in_regions_mem(buf, &regions)?;
    for page_res in results {
        for (i, region) in page_res.regions.iter().enumerate() {
            if region.needs_ocr {
                println!("Page {} region {} → OCR required", page_res.page + 1, i + 1);
            } else {
                println!("Page {} region {} text:\n{}", page_res.page + 1, i + 1, region.text);
            }
        }
    }
    Ok(())
}

Source: src/lib.rs lines 7775-7792

---

Key Source Files Reference

File	Role in Smart Routing
src/lib.rs	Public API entry points: classify_pdf_mem, extract_pages_markdown_mem, OCR flag structures
src/detector.rs	PDF type classification and OCR-need signal generation
src/extractor/content_stream.rs	Low-level text extraction from PDF content streams
src/markdown/analysis.rs	Layout statistics for robust conversion decisions
src/markdown/convert.rs	Markdown rendering with needs_ocr flag respect
src/tables/	Table detection heuristics influencing complexity flags
src/bin/pdf2md.rs	Reference CLI implementation of full routing pipeline

---

Performance Characteristics

• Classification latency: ~10-50ms for typical documents
• Memory efficiency: Single-parse shared representation eliminates redundant copies
• Deterministic routing: No external dependencies or network calls during detection
• Cost reduction: OCR invoked only on pages with verified extraction failures

---

Summary

• classify_pdf_mem provides sub-50ms routing decisions for simple use cases
• extract_pages_markdown_mem combines extraction and routing in one call
• The detection pipeline in src/detector.rs analyzes raster images, vector text, and font encodings to set needs_ocr flags
• Region-based extraction via extract_text_in_regions_mem supports fine-grained pipeline control
• Reference implementation available in src/bin/pdf2md.rs for production deployment patterns

---

Frequently Asked Questions

How accurate is the OCR-need detection in pdf-inspector?

The detection is deterministic and based solely on PDF internal structure analysis. It examines four specific signals: page-level raster images, vector-drawn text presence, broken font encodings (CID garbage), and GID-encoded fonts without Unicode mappings. According to the firecrawl/pdf-inspector source code in src/detector.rs, each flagged page includes a machine-readable ocr_reason explaining which signal triggered the fallback requirement.

Can I use pdf-inspector routing with cloud OCR services like AWS Textract or Google Document AI?

Yes. The needs_ocr boolean flags and page indices produced by extract_pages_markdown_mem can drive any downstream OCR service. Render only the flagged pages to images, then submit those specific pages to your cloud provider. This selective submission pattern typically reduces OCR costs by 60-90% for mixed-content PDF collections containing both scanned and digitally-born documents.

Does pdf-inspector require GPU acceleration for its routing decisions?

No GPU is required for routing. The classification and extraction logic runs entirely on CPU using deterministic parsing of PDF content streams. OCR engines (Tesseract, EasyOCR, or cloud services) are only invoked for pages explicitly flagged by needs_ocr, and those external calls are the responsibility of your application code, not pdf-inspector itself.

What Rust version and dependencies does pdf-inspector require?

The library targets stable Rust and depends on standard PDF parsing crates. Check the Cargo.toml in the firecrawl/pdf-inspector repository for current version requirements. The public API surface in src/lib.rs is designed for FFI compatibility, enabling bindings to Python, Node.js, or other languages through standard tools like PyO3 or napi-rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}