# Hybrid OCR Pipelines in pdf‑inspector: Region‑Based Text Extraction and Layout Models Explained

> Explore hybrid OCR pipelines in pdf-inspector. Learn how region-based text extraction and layout models optimize GPU-accelerated OCR for accurate PDF analysis.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-08-11

---

**pdf‑inspector implements hybrid OCR by first classifying PDFs to identify which pages need OCR, then extracting positioned text once for the entire document, and finally running GPU‑accelerated OCR only on the specific problematic regions.**

The `firecrawl/pdf-inspector` Rust crate solves the expensive problem of OCR‑ing entire PDFs by splitting processing into lightweight classification, region‑based extraction, and targeted OCR fallback. This architecture minimizes GPU usage while maintaining accuracy across scanned documents, mixed PDFs, and natively text‑based files.

## Lightweight PDF Classification Stage

The hybrid pipeline starts with **fast PDF classification** that makes OCR decisions without heavy processing. The `classify_pdf_mem` function in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) (lines 89‑101) peeks at the PDF structure to determine its type and flag pages requiring OCR:

```rust
let classification = pdf_inspector::classify_pdf_mem(&pdf_bytes)?;
println!("Pages needing OCR: {:?}", classification.pages_needing_ocr);

```

This function returns a **`PdfClassification` struct** defined at [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) lines 73‑87, containing:

- `pdf_type` – detected category (`TextBased`, `Scanned`, or `Mixed`)
- `pages_needing_ocr` – zero‑indexed list of pages requiring fallback OCR
- `total_pages` and `confidence` score for the classification

The classification stage completes in approximately **10‑50 milliseconds**, making the OCR decision cheap enough to run on every document.

## Region‑Based Text Extraction Architecture

pdf‑inspector implements **region‑based text extraction** (explicitly noted in the comment at [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) lines 72‑76) to avoid redundant processing. The approach operates in two distinct phases:

### Full‑Document Geometry Extraction

The extractor reads the PDF once and builds complete geometric data through `extract_positioned_text_from_doc`. This captures:

- Positioned text items with precise coordinates
- Bounding rectangles for text regions
- Line geometry and spatial relationships

This single‑pass extraction means layout analysis operates on in‑memory data with near‑zero incremental cost.

### Per‑Page Slicing with OCR Flags

The `extract_pages_markdown_mem` function slices the pre‑computed geometry per page. Each `PageMarkdown` struct (defined at [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) lines 9‑20) includes:

```rust
pub struct PageMarkdown {
    pub page: usize,
    pub markdown: String,
    pub needs_ocr: bool,  // ← Key flag for hybrid pipeline
    // ... layout metadata
}

```

The `needs_ocr` flag triggers when native text is unreliable—specifically for **GID‑encoded fonts**, **broken encodings**, or **garbled vector text**. This region‑based approach ensures only problematic pages proceed to OCR, not entire documents.

## Layout Model Integration

pdf‑inspector embeds **cheap layout detectors** that run during geometric extraction in [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs). These detectors analyze the positioned text and rectangles already held in memory:

- **Table detection** – identifies tabular structures via rectangle‑based, line‑based, and heuristic strategies in `src/tables/`
- **Multi‑column layout detection** – recognizes complex reading orders
- **Spatial analysis** – uses line geometry for structural understanding

Results populate the **`PagesExtractionResult`** struct at [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) lines 22‑38, which carries tables, column information, and OCR flags through the pipeline. Because these models operate on extracted geometry rather than rasterized images, they add negligible overhead.

## The Complete Hybrid Pipeline Flow

The three‑stage hybrid OCR workflow orchestrates native extraction and GPU fallback:

1. **Classify** – Call `classify_pdf_mem` or `process_pdf_with_options` to identify `Scanned` pages and specific `pages_needing_ocr`
2. **Extract with metadata** – Run `extract_pages_markdown_mem`; pages with `needs_ocr == false` yield reliable markdown immediately
3. **Targeted OCR** – Send only flagged page images to GPU engines like PaddleOCR or Tesseract, then merge results

```rust
use pdf_inspector::{classify_pdf_mem, extract_pages_markdown_mem};

fn process_hybrid(pdf_bytes: &[u8]) -> Result<(), pdf_inspector::PdfError> {
    // Stage 1: Lightweight classification
    let classification = classify_pdf_mem(pdf_bytes)?;
    
    // Stage 2: Region-based extraction
    let extraction = extract_pages_markdown_mem(pdf_bytes, None)?;
    
    for page in extraction.pages {
        if page.needs_ocr {
            // Stage 3: GPU OCR for this specific region only
            let ocr_text = run_gpu_ocr_on_page(page.page);
            merge_into_output(ocr_text);
        } else {
            // Native extraction is reliable — use directly
            output_markdown(page.markdown);
        }
    }
    Ok(())
}

```

This region‑based design **avoids sending whole PDFs to OCR engines**, dramatically reducing GPU load and processing time.

## Performance Characteristics

The hybrid architecture achieves efficiency through three key properties:

- **Single extraction pass** – Full‑document geometry computation happens once regardless of page count
- **In‑memory layout analysis** – Table and column detectors reuse positioned data without re‑parsing
- **Surgical OCR invocation** – GPU engines process only regions where native text fails

The result preserves **native text quality** for clean, properly encoded PDFs while maintaining **scanned document support** through targeted fallback.

## Summary

- **`classify_pdf_mem`** in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) performs fast PDF classification (10‑50ms) to identify pages needing OCR
- **Region‑based extraction** builds full‑document geometry once, then slices per‑page with `needs_ocr` flags
- **Layout models** for tables and columns run cheaply on in‑memory geometric data in [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) and `src/tables/`
- **Targeted OCR** sends only problematic regions to GPU engines, avoiding whole‑document processing
- The `PageMarkdown` and `PagesExtractionResult` structs carry extraction metadata through the pipeline

## Frequently Asked Questions

### How does pdf‑inspector decide which pages need OCR?

The `classify_pdf_mem` function analyzes PDF structure in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) and returns `pages_needing_ocr` as a list of zero‑indexed page numbers. It detects GID‑encoded fonts, broken encodings, and vector‑only content that native extraction cannot reliably decode. The classification runs in milliseconds without rendering pages.

### What makes the layout detection "cheap" in pdf‑inspector?

Layout detectors operate on positioned text items, rectangles, and line geometry already extracted into memory by `extract_positioned_text_from_doc`. Because this geometric data exists in RAM, table and column detection in `src/tables/` and [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) requires no additional PDF parsing or image rendering—unlike OCR‑based layout analysis that must rasterize pages first.

### Can I use pdf‑inspector with custom OCR engines?

Yes. The `extract_pages_markdown_mem` function returns per‑page `needs_ocr` flags and page indices. Your application sends only flagged pages to any GPU OCR engine (PaddleOCR, Tesseract, cloud APIs), then merges the OCR text back into the markdown output. The crate intentionally separates extraction metadata from OCR execution for this flexibility.

### Where does the actual OCR execution happen in pdf‑inspector?

The crate itself does not include OCR engines. According to the source code in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs), it provides classification and extraction metadata that tells calling code *which* regions need OCR. The caller then routes those specific page images to their chosen GPU OCR implementation—enabling hybrid pipelines without bundling heavy OCR dependencies.