How Per‑Page OCR Routing Works with `pagesNeedingOcr` in PDF‑Inspector

Per‑page OCR routing in PDF‑Inspector uses a detection‑driven bitmap to minimize OCR costs: the detector classifies each page and returns a Vec<bool> flagging only the pages that need OCR, then the extraction orchestrator routes those specific pages to the OCR pipeline while extracting the rest normally.

PDF‑Inspector is a Rust‑based extraction engine from firecrawl/pdf‑inspector that converts PDFs to clean Markdown. A core architectural challenge is handling mixed PDFs—documents containing both searchable text and scanned image pages. Sending an entire document to OCR is expensive and slow. Instead, PDF‑Inspector implements selective per‑page OCR routing through the pagesNeedingOcr mechanism, which identifies exactly which pages require OCR before any heavy processing begins.

---

PDF Type Detection: The pagesNeedingOcr Bitmap

The routing logic begins in the detection phase. The detect_pdf_type function in src/detector.rs scans every page to classify the document into one of three PdfType variants:

Variant	Description
TextBased	All pages contain extractable Unicode text; no OCR needed.
Scanned	All pages are image‑only; full document OCR required.
Mixed(Vec)	Hybrid document where the Vec<bool> serves as the pagesNeedingOcr bitmap—true marks pages needing OCR, false marks pages with searchable text.

This classification enables the extractor to treat each page individually rather than applying a one‑size‑fits‑all strategy.

---

Using pagesNeedingOcr in the Extraction Orchestrator

When the detector returns Mixed, the extraction orchestrator in src/extractor/mod.rs uses the bitmap to make per‑page routing decisions:

1. Check the flag for each page in the pagesNeedingOcr vector.
2. If false: Route to the standard content‑stream parser (src/extractor/content_stream.rs) for fast, accurate text extraction.
3. If true: Route to the OCR pipeline (via external tool or integrated OCR crate) to generate searchable text from the page image.

This selective routing ensures OCR latency and compute costs are incurred only where necessary.

---

Code Examples

CLI Usage with Automatic Per‑Page OCR

The pdf2md binary automatically respects the pagesNeedingOcr detection without manual flagging:

# OCR runs only on scanned pages; text pages extract normally

pdf2md mixed_invoice.pdf --output invoice.md

Programmatic Access to pagesNeedingOcr

Access the per‑page bitmap directly when using PDF‑Inspector as a library:

use pdf_inspector::{process_pdf_with_options, PdfType, ProcessOptions};

let opts = ProcessOptions::default();
let result = process_pdf_with_options("mixed_report.pdf", opts).unwrap();

match result.pdf_type {
    PdfType::Mixed(pages_needing_ocr) => {
        for (page_num, needs_ocr) in pages_needing_ocr.iter().enumerate() {
            println!("Page {} needs OCR: {}", page_num + 1, needs_ocr);
        }
    }
    PdfType::TextBased => println!("No OCR required"),
    PdfType::Scanned => println!("Full document OCR required"),
}

---

Key Source Files

File	Purpose
[src/detector.rs](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)	Implements detect_pdf_type and builds the PdfType::Mixed(Vec<bool>) bitmap.
[src/extractor/mod.rs](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs)	Orchestrates extraction; consumes pagesNeedingOcr to route pages appropriately.
[src/extractor/content_stream.rs](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs)	Standard PDF text extraction for non‑OCR pages.
[src/markdown/convert.rs](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/convert.rs)	Assembles final Markdown from mixed extraction and OCR results.
[src/bin/pdf2md.rs](https://github.com/firecrawl/pdf-inspector/blob/main/src/bin/pdf2md.rs)	CLI entry point that wires detection through to extraction.

---

Summary

• Detection first: src/detector.rs identifies which pages lack searchable text and returns a pagesNeedingOcr bitmap via PdfType::Mixed.
• Selective routing: src/extractor/mod.rs uses the bitmap to send only flagged pages to OCR, keeping text‑based pages on the fast path.
• Cost efficiency: Per‑page granularity eliminates wasteful full‑document OCR and preserves extraction quality across heterogeneous documents.
• Transparent integration: Both the pdf2md CLI and the Rust library API handle pagesNeedingOcr automatically.

---

Frequently Asked Questions

What happens if a PDF is classified as Scanned instead of Mixed?

When detect_pdf_type returns PdfType::Scanned, every page is treated as image‑only. The extractor skips the normal text pipeline and sends all pages to OCR as a batch, since no bitmap is needed—every flag would be true.

Can I force OCR on specific pages even if the detector flags them as text?

PDF‑Inspector does not expose a per‑page override in the current API. The routing decision is deterministic based on the detector's analysis of Unicode text presence in the content stream. For custom workflows, you would need to pre‑process the PDF or modify the detection logic in src/detector.rs.

How does pagesNeedingOcr impact performance on large documents?

The performance benefit is substantial: pages marked false are parsed via content_stream.rs in microseconds, while OCR pages incur seconds per page. A 100‑page document with 5 scanned pages processes ~95% of content at native speed, with OCR costs isolated to the minority that need it.

Where is the OCR engine itself implemented?

PDF‑Inspector's core repository focuses on detection and routing. The actual OCR execution (image‑to‑text) is delegated to external engines or optional companion crates. The pagesNeedingOcr bitmap provides the integration point for plugging in Tesseract, cloud OCR APIs, or other backends from src/extractor/mod.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}