How pdf-inspector Classifies PDF Types: TextBased, Scanned, ImageBased, and Mixed

pdf-inspector classifies PDFs by sampling content streams for text operators (Tj/TJ), image objects, and font-encoding signals, then applying a heuristic decision tree in src/detector.rs to categorize documents into four distinct types that determine OCR requirements.

The firecrawl/pdf-inspector repository implements a Rust-based detection pipeline that inspects PDF structure without fully rendering pages. By analyzing text operators, image XObjects, and font metadata across sampled pages, it determines whether a document is natively text-based, scanned, image-heavy, or a mixed hybrid requiring selective OCR.

The Four PDF Classifications

pdf-inspector assigns exactly one PdfType variant to each document based on content analysis:

• TextBased — Sampled pages contain sufficient extractable text (≥ 60% of pages by default) and are not dominated by images. No OCR is required.
• Scanned — No extractable text operators exist; content consists only of images or vector-outlined text. OCR is required for the entire document.
• ImageBased — No extractable text, but the document contains images alongside a small amount of vector text (e.g., diagrams). OCR is required for the entire document.
• Mixed — The PDF contains meaningful text coexisting with image-heavy pages or template backgrounds where images provide essential context. OCR is recommended only for specific flagged pages.

These classifications are defined in the PdfType enum at lines 14–22 of src/detector.rs.

Detection Configuration and Sampling Strategy

Classification behavior is controlled by the DetectionConfig struct (lines 70–88), which exposes tunable thresholds:

pub struct DetectionConfig {
    pub strategy: ScanStrategy,          // Sampling approach
    pub min_text_ops_per_page: u32,        // Minimum Tj/TJ operators to count as "text"
    pub text_page_ratio_threshold: f32,  // Default: 0.6 (60% of pages must have text)
}

The ScanStrategy enum determines which pages are analyzed in detect_from_document (lines 1912–2000):

• EarlyExit — Scan every page sequentially, stopping at the first non-text page.
• Full — Analyze all pages regardless of length.
• Sample(N) — Select N evenly-spaced pages (first, last, and distributed middle pages) for performance on large documents.
• Pages(vec) — Scan an explicit user-provided list of page indices.

Per-Page Content Analysis

For each sampled page, analyze_page_content extracts structural metrics cached for later phases (lines 2120–2240):

Metric	Detection Purpose
text_operator_count	Count of Tj/TJ operators indicating raw extractable text
has_images / image_count	Presence of XObject image operators
has_template_image	Single large background image (typical of scanned PDFs with OCR overlay)
has_vector_text	Path-drawing operations suggesting text rendered as vectors
has_identity_h_no_tounicode / has_only_type3_fonts	Fonts that cannot decode to Unicode (garbled text signals)
has_decodable_text_fonts	CID-encoded fonts that are properly decodable (prevents misclassification)

The Classification Decision Tree

After sampling, pdf-inspector calculates a text-ratio (pages_with_text / pages_sampled) and evaluates the decision tree at lines 3100–3340:

if has_template_images && pages_with_text > 0 {
    // Template PDFs mix text with essential background images
    (PdfType::Mixed, 0.5 + 0.3 * (1.0 - template_ratio))
} else if text_ratio >= config.text_page_ratio_threshold {
    (PdfType::TextBased, text_ratio)
} else if pages_with_text == 0 && (pages_with_images > 0 || pages_with_vector_text > 0) {
    if total_text_ops == 0 && pages_with_vector_text == 0 {
        (PdfType::Scanned, 0.95)
    } else {
        (PdfType::ImageBased, 0.8)
    }
} else if pages_with_text > 0 && (pages_with_images > 0 || pages_with_vector_text > 0) {
    (PdfType::Mixed, 0.7)
} else if total_text_ops == 0 {
    (PdfType::Scanned, 0.9)
} else {
    (PdfType::TextBased, text_ratio.max(0.5))
}

Key logic branches:

• Mixed triggers when template images coexist with text, or when both text and images appear across the document.
• Scanned is the fallback when zero text operators are detected.
• ImageBased serves as a middle ground when vector text exists but image content dominates.

Mixed-Type OCR Targeting and Newspaper Detection

Even TextBased documents may trigger OCR recommendations. At lines 3500–3570, pdf-inspector detects newspaper-style layouts where high text-operator counts combine with low font-change ratios, setting ocr_recommended to true.

For Mixed classifications, Phase 2 (lines 3830–4160) revisits all pages to build a specific OCR list:

if (analysis.has_template_image && looks_like_scan) ||
    analysis.has_vector_text ||
    (analysis.text_operator_count < config.min_text_ops_per_page && analysis.has_images) {
    ocr_pages.push(page_num);
}

Phase 3 (lines 4200–4240) additionally flags pages with undecodable fonts (Identity-H without ToUnicode or Type 3-only fonts). Each flagged page receives a human-readable reason (scanned, no_text, vector_text, suspected_garbled_text) via page_ocr_reasons (lines 5250–5275).

Implementation Examples

Rust Library Usage

use pdf_inspector::{detect_pdf_type, PdfType};

fn main() -> Result<(), pdf_inspector::PdfError> {
    // Detect using default configuration (Sample(8) strategy)
    let result = detect_pdf_type("document.pdf")?;
    
    match result.pdf_type {
        PdfType::TextBased => println!("Native text document"),
        PdfType::Mixed => println!("Mixed content: OCR needed for {:?}", result.pages_needing_ocr),
        _ => println!("OCR required"),
    }
    Ok(())
}

Python Bindings

from pdf_inspector import classify_pdf

result = classify_pdf("document.pdf")
print(f"Type: {result.pdf_type}")  # "text_based", "scanned", "image_based", or "mixed"

print(f"OCR pages: {result.pages_needing_ocr}")
print(f"Reasons: {result.ocr_reasons_by_page}")

CLI Command

$ detect-pdf report.pdf
MIXED (some pages need OCR)
  OCR pages: 2, 5, 7
  Reasons:
    2 → scanned
    5 → vector_text
    7 → suspected_garbled_text

Summary

• pdf-inspector classifies PDFs into TextBased, Scanned, ImageBased, or Mixed types using a sampling strategy defined in DetectionConfig.
• The core algorithm in src/detector.rs (lines 3100–3340) evaluates text operator ratios, image presence, and font decodability to select the appropriate classification.
• Mixed documents undergo Phase 2 analysis to identify specific pages requiring OCR, while Scanned and ImageBased types trigger full-document OCR recommendations.
• All public interfaces—Rust (detect_pdf_type), Python (classify_pdf), and CLI (detect-pdf)—return the same structured result containing the PDF type, confidence score, and per-page OCR recommendations.

Frequently Asked Questions

What is the difference between Scanned and ImageBased PDFs in pdf-inspector?

Scanned PDFs contain absolutely no text operators and no vector text—only raster images or image masks. ImageBased PDFs contain no extractable text operators but do include vector-drawn text or diagrams alongside images. According to the source code at lines 3118–3124, ImageBased is selected when total_text_ops == 0 but pages_with_vector_text > 0, whereas Scanned requires both counts to be zero.

How does pdf-inspector handle template-based PDFs with background images?

Template PDFs—documents with a single large background image behind text layers—are classified as Mixed rather than TextBased. The detector identifies has_template_image during per-page analysis and triggers the Mixed classification at line 3110 when template images coexist with text content, ensuring the background image is processed via OCR if it contains essential visual information.

Can I customize the sampling strategy for PDF type detection?

Yes. The DetectionConfig struct accepts a ScanStrategy variant that controls page selection. You can force analysis of all pages with ScanStrategy::Full, use ScanStrategy::EarlyExit to stop at the first non-text page for speed, or provide exact page numbers with ScanStrategy::Pages(vec![1, 5, 10]). This configuration is passed to detect_pdf_type or detect_pdf_type_mem in the Rust API.

Why does a TextBased PDF still recommend OCR in some cases?

pdf-inspector implements newspaper-layout detection at lines 3500–3570 that flags dense, multi-column text layouts as ocr_recommended even when the classification remains TextBased. This occurs when pages show high text-operator counts but low font-change diversity, indicating complex columnar layouts where standard text extraction may fail to preserve reading order.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}