PDF Type Detection Strategies in firecrawl/pdf-inspector: Implementation and Usage

The firecrawl/pdf-inspector library classifies PDFs into TextBased, Scanned, ImageBased, or Mixed types using the ScanStrategy enum in src/detector.rs to control which pages are analyzed and when to stop scanning.

The firecrawl/pdf-inspector repository provides a Rust-based solution for automatically categorizing PDF documents based on their internal structure. By implementing configurable PDF type detection strategies, the library enables precise routing of documents to appropriate extraction pipelines. At the core of this system lies the ScanStrategy enum defined in src/detector.rs, which determines the granularity of page inspection and directly impacts both performance and classification accuracy.

The Four ScanStrategy Variants for PDF Type Detection

The detector supports four distinct scanning approaches defined in the ScanStrategy enum at lines 25-40 of src/detector.rs. Each variant determines the sample_indices list and the allow_early_exit flag (see the match block at lines 93-101), allowing callers to balance speed against detection precision.

EarlyExit: Fast Text Detection

EarlyExit scans pages sequentially and stops immediately upon encountering the first non-text page. This strategy sets allow_early_exit to true and is optimized for pipelines that need to quickly route pure-text PDFs to fast extractors. According to the source code, this is the default choice for small PDFs where processing speed outweighs the need for absolute classification precision.

Full: Comprehensive Page Analysis

Full examines every page in the document without early termination, guaranteeing the most accurate distinction between Mixed and Scanned types. This strategy disables early exit by setting allow_early_exit to false and is recommended for large documents containing interleaved text and images where reliable mix detection is critical for downstream processing.

Sample: Statistical Page Sampling

Sample(u32) selects up to N evenly distributed pages throughout the document, including the first, middle, and last pages. This approach balances speed and accuracy for very large PDFs, allowing the caller to trade precision for performance by adjusting the sample size parameter based on document length and confidence requirements.

Pages: Targeted Explicit Inspection

Pages(Vec<u32>) analyzes only the specific 1-indexed page numbers provided by the caller. This strategy is useful when the caller possesses prior knowledge about representative pages—such as cover pages or typical content sections—and wants to minimize unnecessary processing while maintaining high confidence in the classification result.

Two-Phase Detection Pipeline Implementation

After building the sample_indices list based on the selected ScanStrategy, the detector executes a two-phase classification pipeline implemented in src/detector.rs.

Phase 1: Page-Level Content Analysis

The first phase inspects individual pages for text operators, embedded images, vector text, font characteristics, and template-image presence. As implemented in lines 336-378, this phase specifically flags newspaper-style layouts—documents exhibiting high text density but low Tf/Tj operator ratios—which may trigger OCR recommendations even for otherwise TextBased PDFs to preserve proper reading order.

Phase 2: OCR Recommendation Logic

The second phase aggregates metrics including text ratio, template-image ratio, and undecodable font presence to determine the final PdfType and whether OCR is recommended. Lines 310-335 contain the decision logic that evaluates these aggregated document characteristics to produce the final PdfTypeResult.

Configuring PDF Type Detection in Rust

The DetectionConfig struct and detect_pdf_type_with_config function in src/lib.rs provide the public API for utilizing these strategies with explicit configuration.

use pdf_inspector::detector::{detect_pdf_type, DetectionConfig, ScanStrategy};

// Default strategy using Sample(8)
let result = detect_pdf_type("reports/annual.pdf")?;
println!("Detected: {:?}, OCR? {}", result.pdf_type, result.ocr_recommended);

// Full scan for maximum accuracy on mixed content
let config = DetectionConfig {
    strategy: ScanStrategy::Full,
    ..Default::default()
};
let result = pdf_inspector::detector::detect_pdf_type_with_config("books/large.pdf", config)?;
println!("Pages sampled: {}", result.pages_sampled);

// Custom sample of 5 pages for quick heuristic
let config = DetectionConfig {
    strategy: ScanStrategy::Sample(5),
    ..Default::default()
};
let result = detect_pdf_type_with_config("manuals/manual.pdf", config)?;
println!("Confidence: {:.2}", result.confidence);

// Explicit page selection for targeted analysis
let config = DetectionConfig {
    strategy: ScanStrategy::Pages(vec![1, 42]),
    ..Default::default()
};
let result = detect_pdf_type_with_config("contracts/contract.pdf", config)?;
assert!(result.pages_sampled == 2);

Integration with Extraction Pipelines

The PdfTypeResult output drives downstream processing decisions in src/extractor/mod.rs, which orchestrates whether to apply OCR or direct text extraction based on the detected type. Additionally, src/tables/detect_heuristic.rs consumes the classification to apply specialized table-detection heuristics, demonstrating how the detection strategy influences the entire document processing workflow.

Summary

• Four classification types: TextBased, Scanned, ImageBased, and Mixed, determined by content stream analysis and layout heuristics.
• Configurable strategies: The ScanStrategy enum offers EarlyExit, Full, Sample(N), and Pages variants to balance speed versus accuracy based on document characteristics.
• Implementation location: Core logic resides in src/detector.rs, with sampling configuration handled at lines 93-101 and classification logic spanning lines 310-378.
• Two-phase analysis: Page-level inspection followed by aggregated OCR recommendation ensures robust detection of complex layouts including newspaper-style documents.
• Public API: detect_pdf_type and detect_pdf_type_with_config in src/lib.rs provide flexible Rust interfaces for strategy configuration and execution.

Frequently Asked Questions

What is the default PDF type detection strategy in pdf-inspector?

The default strategy uses ScanStrategy::Sample(8), which examines up to eight evenly distributed pages throughout the document. This provides a reasonable balance between processing speed and detection accuracy for most standard PDFs without requiring explicit configuration from the caller.

When should I use the Full scan strategy instead of Sample?

Use ScanStrategy::Full when processing large documents suspected of containing mixed content types—such as interleaved text pages and image pages—where missing a single non-text page could result in incorrect pipeline routing. This strategy scans every page at the cost of increased processing time, ensuring reliable distinction between Mixed and Scanned classifications according to the implementation in src/detector.rs.

How does pdf-inspector handle newspaper-style layouts?

The detector identifies newspaper-style layouts during Phase 1 analysis (lines 336-378) by detecting high text density combined with low Tf/Tj operator ratios. Even if such documents technically contain extractable text, the system may recommend OCR processing to better handle complex columnar layouts and preserve proper reading order across multiple columns.

Can I specify exactly which pages to analyze for type detection?

Yes, the ScanStrategy::Pages(Vec<u32>) variant accepts a vector of 1-indexed page numbers, allowing precise control over which pages undergo inspection. This is particularly efficient when you know representative pages in advance, such as analyzing page 1 (cover) and page 42 (standard content) to characterize the entire document without scanning intermediate pages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}