# How `extract_pages_markdown_mem` Enables Per-Page Extraction with Hybrid OCR Routing in PDF Inspector

> Learn how extract_pages_markdown_mem in pdf-inspector unlocks per page markdown extraction with hybrid OCR routing for reliable text quality.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: internals
- Published: 2026-08-14

---

**`extract_pages_markdown_mem` is the core API in firecrawl/pdf-inspector that returns individual page Markdown with granular OCR flags, using the same classifier signals as the document-level pipeline to route pages through OCR when text quality is unreliable.**

The `firecrawl/pdf-inspector` Rust crate provides two primary extraction paths: `process_pdf_mem` for simple full-document strings, and `extract_pages_markdown_mem` for per-page results with hybrid OCR routing. This article explains how the latter implements **per-page extraction** and **hybrid OCR routing**— ensuring that extraction decisions stay synchronized with the document classifier.

## What `extract_pages_markdown_mem` Returns

Unlike the single-string API, `extract_pages_markdown_mem` returns a `PagesExtractionResult` containing a `Vec<PageMarkdown>` in [[`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs#L8-L20). Each `PageMarkdown` struct includes:

- **`page`**: Zero-based page index
- **`markdown`**: Extracted Markdown content for that page
- **`needs_ocr`**: Boolean flag indicating OCR is recommended
- **`ocr_reason`**: Optional string explaining why (e.g., "scanned", "vector_text")

This structure lets downstream pipelines process only flagged pages through OCR services, avoiding wasted GPU cycles on clean digital PDFs.

## Hybrid OCR Routing: Shared Signals with the Classifier

The key architectural decision is **signal reuse**: `extract_pages_markdown_mem` uses the same detection logic as `detect_pdf_type` to ensure consistency. According to the source code, three signal categories determine the `needs_ocr` flag per page:

### 1. Text Quality Signals

From [[`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs), the function `analyze_text_quality` runs once per document and identifies:

- **GID-encoded fonts**: Character glyphs encoded as graphics IDs rather than proper Unicode
- **Garbled or empty text streams**: Extracted text that fails entropy or validity checks

These map to `OCR_REASON_VECTOR_TEXT` or `OCR_REASON_SUSPECTED_GARBLED_TEXT`.

### 2. Page-Level Visual Signals

From [[`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs#L1820-L1835), the `page_ocr_signals` function inspects each page for:

- **`has_template_image`**: Large background images indicating scanned documents
- **`has_vector_text`**: Text drawn as vector paths without proper text objects

These map to `OCR_REASON_SCANNED` or `OCR_REASON_VECTOR_TEXT`.

### 3. Document-Wide Context

Font statistics, layout complexity (tables, columns), and header/footer patterns are computed **once for the entire document** before per-page slicing. This guarantees that:

- Column detection thresholds remain consistent across page ranges
- Table cell preservation rules apply uniformly
- Page number removal masks work correctly for any subset

## Step-by-Step Extraction Flow

The implementation in [[`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) lines 57-78](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs#L57-L78) follows this pipeline:

1. **Validation**: `validate_pdf_bytes` and `load_document_from_mem` parse the PDF structure
2. **Global analysis**: Extract all items with `extract_positioned_text_for_document_analysis` to build font stats and `text_quality` evaluation via `analyze_text_quality`
3. **Page filtering**: Convert the `pages` argument to a `HashSet<u32>` for O(1) lookup
4. **Layout computation**: `compute_layout_complexity_with_chart_regions` and `calculate_font_stats_from_items` establish document-wide baselines
5. **Per-page processing**: For each requested page:
   - Slice items, rectangles, and removal masks for that page
   - **Combine OCR signals**: `has_gid`, `has_text_quality_issue`, and `page_ocr_signals` results
   - Set `needs_ocr` and `ocr_reason` if any signal fires
   - Generate Markdown via `to_markdown_from_items_with_rects_and_lines` with document-wide font stats
   - Collect into `PageMarkdown`

This prevents the silent mismatch described in issue #227—where the classifier would route to OCR but extraction would proceed with garbled text.

## Code Examples

### Rust: Direct Crate Usage

```rust
use pdf_inspector::{extract_pages_markdown_mem, PagesExtractionResult};

fn main() -> Result<(), pdf_inspector::PdfError> {
    // Load PDF into memory
    let data = std::fs::read("example.pdf")?;

    // Extract all pages (None = every page)
    let result: PagesExtractionResult = extract_pages_markdown_mem(&data, None)?;

    for page in result.pages {
        println!("--- Page {} ---", page.page + 1);
        println!("{}", page.markdown);
        if page.needs_ocr {
            println!("⚠️  OCR required: {}", 
                page.ocr_reason.unwrap_or("unknown".into()));
        }
    }
    Ok(())
}

```

### Python: PyO3 Bindings

```python
import pdf_inspector

with open("example.pdf", "rb") as f:
    data = f.read()

# Extract specific pages with zero-based indexing

result = pdf_inspector.extract_pages_markdown_bytes(data, pages=[0, 2, 5])

for page in result.pages:
    print(f"\n--- Page {page.page + 1} ---")
    print(page.markdown)
    if page.needs_ocr:
        print(f"⚠️  Route to OCR: {page.ocr_reason}")

```

Both examples demonstrate how callers receive actionable per-page metadata—enabling selective OCR pipelines that process only `needs_ocr == true` pages.

## Key Source Files

| File | Purpose |
|------|---------|
| [[`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) | `extract_pages_markdown_mem` implementation, `PageMarkdown`/`PagesExtractionResult` definitions |
| [[`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) | `page_ocr_signals` and `analyze_page_images` for visual OCR signals |
| [[`src/markdown/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/mod.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/mod.rs) | `to_markdown_from_items_with_rects_and_lines` for Markdown generation |
| [[`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) | Low-level positioned text and layout extraction |
| [[`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs)](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs) | GID-encoded font detection and garbled text analysis |

## Summary

- **`extract_pages_markdown_mem`** returns granular per-page results with OCR routing metadata
- **Hybrid OCR routing** reuses `detect_pdf_type` signals: `has_template_image`, `has_vector_text`, and text-quality issues
- **Document-wide preprocessing** ensures consistent font stats, layout detection, and header/footer removal across any page subset
- **Signal consistency** between extraction and classification prevents the mismatches that caused issue #227

## Frequently Asked Questions

### What is the difference between `process_pdf_mem` and `extract_pages_markdown_mem`?

`process_pdf_mem` returns a single concatenated Markdown string for the entire document with basic OCR routing, while `extract_pages_markdown_mem` returns structured per-page results with individual `needs_ocr` flags and reasons. Use the latter when you need granular control over which pages to process through OCR services.

### How does the hybrid OCR routing decide if a page needs OCR?

The routing combines three signal layers: **text quality analysis** (GID fonts, garbled text from `analyze_text_quality`), **visual signals** (template images, vector text from `page_ocr_signals`), and **document context** (global font statistics). Any triggering signal sets `needs_ocr = true` with an appropriate reason constant.

### Can I extract only specific pages and still get accurate OCR flags?

Yes. The `pages` parameter accepts a `Vec<u32>` or `None` for all pages. Document-wide preprocessing (font stats, layout complexity, header/footer detection) runs once before slicing, so any subset receives consistent OCR routing decisions based on complete document context.

### Why is signal reuse with `detect_pdf_type` important?

Reusing the same signals prevents classification-extraction mismatches where the document classifier would route to OCR but the extraction pipeline would proceed with unreliable text. As noted in source comments referencing issue #227, this synchronization eliminates silent failures on scanned or vector-text PDFs.