How pdf-inspector Classifies PDFs: Architecture, Configuration, and Code Examples

pdf-inspector determines whether a PDF is TextBased, Scanned, ImageBased, or Mixed by sampling pages and counting text operators, without loading the entire document.

The pdf-inspector library from Firecrawl provides fast, lightweight PDF classification for high-throughput pipelines. This article explains precisely how the classification system works based on the source code implementation, with runnable examples you can adapt immediately.

Core Detection Architecture

The classification logic lives primarily in src/detector.rs, which exposes two main entry points:

• detect_pdf_type(path) – runs detection with default configuration
• detect_pdf_type_with_config(path, config) – allows custom scan strategies and thresholds

Both functions return a PdfTypeResult containing the classification, confidence metrics, and OCR recommendations. The implementation avoids full document parsing by using page sampling heuristics [source lines 92-100].

Scan Strategy: Four Modes for Speed vs. Accuracy

The ScanStrategy enum defines how pages are inspected [source lines 25-40]:

Strategy	Behavior	Best For
EarlyExit	Stop at first non-text page	Fast rejection of scanned documents
Full	Inspect every single page	Maximum accuracy, slower processing
Sample(N)	Check N evenly-distributed pages (default: 8)	Balanced speed and precision
Pages(vec)	Only check user-specified page numbers	Targeted analysis

The default Sample(8) strategy provides optimal throughput for most use cases while maintaining reliable classification accuracy.

Per-Page Text Detection

For each selected page, pdf-inspector parses the PDF content stream using lopdf and counts text operators (Tj and TJ). A page qualifies as text-rich when the count exceeds min_text_ops_per_page (default: 3) [source lines 73-76].

This operator-based approach is more reliable than simple font detection because it directly measures actual text rendering commands in the content stream.

Classification Rules and Thresholds

After sampling, the detector builds aggregate metrics in PdfTypeResult [source lines 44-66]:

• total_pages – document page count
• pages_sampled – pages actually inspected
• pages_with_text – pages exceeding text operator threshold
• confidence – calculated reliability of the classification
• ocr_recommended – boolean flag for downstream processing
• pages_needing_ocr – specific page numbers requiring OCR

The final PDF type derives from the ratio pages_with_text / total_pages compared against text_page_ratio_threshold (default: 0.6):

• TextBased – ratio ≥ 0.6
• Scanned – ratio = 0 (no text operators found)
• Mixed – ratio between 0 and threshold (partial text)
• ImageBased – very low ratio with high image page count

OCR Recommendation System

When pages lack sufficient text operators, pdf-inspector sets ocr_recommended = true and records specific page numbers in pages_needing_ocr. This enables downstream pipelines to invoke OCR only where necessary, avoiding expensive full-document processing.

Code Examples

Basic PDF Classification with Defaults

use pdf_inspector::detector::{detect_pdf_type, PdfType};

fn main() -> Result<(), pdf_inspector::PdfError> {
    let result = detect_pdf_type("example.pdf")?;
    
    println!("PDF type: {:?}", result.pdf_type);
    
    if result.ocr_recommended {
        println!("OCR needed on pages: {:?}", result.pages_needing_ocr);
    }
    
    Ok(())
}

Custom Configuration for Maximum Accuracy

use pdf_inspector::detector::{
    detect_pdf_type_with_config, DetectionConfig, ScanStrategy, PdfType,
};

fn main() -> Result<(), pdf_inspector::PdfError> {
    let cfg = DetectionConfig {
        strategy: ScanStrategy::Full,
        min_text_ops_per_page: 5,
        text_page_ratio_threshold: 0.7,
    };
    
    let result = detect_pdf_type_with_config("large.pdf", cfg)?;
    println!("Detected type: {:?}", result.pdf_type);
    
    Ok(())
}

Command-Line Usage

# Output structured JSON for pipeline integration

detect-pdf --json path/to/document.pdf

Key Source Files

File	Purpose	Link
src/detector.rs	Core detection logic, ScanStrategy, DetectionConfig, PdfTypeResult	view
src/lib.rs	Public API re-exports for library consumers	view
src/bin/detect_pdf.rs	CLI binary with --json flag support	view
src/extractor/content_stream.rs	Content stream parser, text operator counting	view
src/extractor/mod.rs	Low-level PDF object handling coordination	view

Summary

• pdf-inspector classifies PDFs using lightweight page sampling rather than full document loading
• ScanStrategy offers four modes from EarlyExit to Full scans, with Sample(8) as the balanced default
• Classification relies on counting Tj/TJ text operators in content streams, with configurable thresholds
• The system returns detailed PdfTypeResult metrics including per-page OCR recommendations
• All detection logic resides in src/detector.rs with clear separation between configuration, sampling, and classification phases

Frequently Asked Questions

How accurate is pdf-inspector's classification compared to OCR-based approaches?

pdf-inspector achieves high accuracy for document routing decisions while being orders of magnitude faster than OCR. The default Sample(8) strategy correctly identifies document types in typical scenarios because text-rich pages distribute evenly in most PDFs. For critical applications requiring certainty, switch to ScanStrategy::Full to inspect every page.

Can I customize what counts as a "text-rich" page?

Yes. The DetectionConfig struct exposes min_text_ops_per_page to adjust the text operator threshold. Increase this value for documents with heavy header/footer metadata you want to ignore, or decrease it for documents with sparse text layouts.

What happens if a PDF contains both scanned and text-based pages?

The classifier returns Mixed when the text page ratio falls between 0 and the threshold (default 0.6). The pages_needing_ocr field specifically identifies which pages lack sufficient text operators, enabling targeted OCR rather than processing the entire document.

Is pdf-inspector suitable for real-time document processing pipelines?

Absolutely. The design prioritizes speed through lazy page sampling and avoids full document materialization. According to the firecrawl/pdf-inspector source code, even ScanStrategy::Full processes only content streams rather than rendering pages, maintaining throughput suitable for high-volume automated systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}