How `extract_text_in_regions_mem` Handles Overlapping PDF Regions

When bounding boxes overlap, extract_text_in_regions_mem assigns each text item exclusively to the region with the largest spatial overlap area, preventing duplicate content while preserving text integrity.

The extract_text_in_regions_mem function in the firecrawl/pdf-inspector repository solves the complex challenge of extracting text from user-defined bounding boxes that may intersect on the same PDF page. When regions overlap, the algorithm must prevent text duplication while ensuring no content is lost during the extraction process. This Rust implementation uses deterministic spatial overlap calculations to assign each text fragment to exactly one region based on geometric priority.

The Three-Stage Overlap Resolution Algorithm

The function implements a sophisticated three-stage pipeline in src/lib.rs to resolve overlapping regions deterministically.

Stage 1: Converting Regions to Bounds

First, the algorithm builds geometric bounds for every specified region. Each input rectangle is converted into a RegionBounds object that accounts for page height transformations and possible page rotation. This normalization ensures consistent coordinate systems before overlap calculations begin. The bounds construction logic resides in lines 78‑92 of src/lib.rs.

Stage 2: Exclusive Assignment by Largest Overlap

The core resolution logic iterates once over all extracted TextItems to perform exclusive assignment. For each text item, the function checks every region using region_overlaps_item to detect intersections. When an overlap exists, region_item_overlap_area calculates the exact spatial intersection area.

The item is assigned to the region with the largest overlap area, stored in a best variable during iteration. The function maintains a region_items vector indexed by region_index to store assigned items, while a parallel had_candidates array records which regions were touched by any item during the scan—even if the item was ultimately assigned elsewhere. This stage is implemented in lines 94‑118 of src/lib.rs.

Stage 3: Post-Processing and OCR Flagging

After assignment, the function assembles text for each region and determines OCR requirements. If a region ends up empty because all its items were claimed by a neighboring region with larger overlap, the code checks the lost_to_neighbor condition (lines 42‑48). When this occurs and the region previously overlapped items (indicated by had_candidates[region_idx]), the region does not trigger OCR. This prevents false positive OCR requests when the empty result is intentional rather than a text extraction quality problem. This final processing occurs in lines 120‑149 of src/lib.rs.

Implementation Details in src/lib.rs

The overlap resolution relies on several key components working together:

• RegionBounds – Normalized geometric representation of user-specified rectangles
• region_overlaps_item – Collision detection between text items and region bounds
• region_item_overlap_area – Precise area calculation for overlapping regions
• had_candidates – Boolean tracking array preventing unnecessary OCR triggers
• lost_to_neighbor – Logic identifying regions that lost content to larger overlaps

The net effect eliminates the previous behavior where overlapping regions duplicated whole lines—a problem that affected approximately 21% of benchmark PDFs in earlier versions.

Practical Usage Example

The following Rust code demonstrates extraction from overlapping regions:

use pdf_inspector::{extract_text_in_regions_mem, RegionText, PageRegionResult};

fn main() -> Result<(), pdf_inspector::PdfError> {
    // Load a PDF into memory
    let pdf_bytes = std::fs::read("sample.pdf")?;

    // Define two overlapping regions on page 0 (top-left origin, points)
    // Region A: (0,0) – (300,300)
    // Region B: (200,200) – (500,500)  // overlaps with A
    let regions = vec![
        (0u32, vec![[0.0, 0.0, 300.0, 300.0], [200.0, 200.0, 500.0, 500.0]]),
    ];

    // Extract
    let results: Vec<PageRegionResult> = extract_text_in_regions_mem(&pdf_bytes, &regions)?;

    // Print each region's text and OCR flag
    for page in results {
        for (i, region) in page.regions.iter().enumerate() {
            println!("Page {} – Region {}:", page.page + 1, i + 1);
            println!("  text: {}", region.text);
            println!("  needs OCR: {}", region.needs_ocr);
        }
    }

    Ok(())
}

Items falling within the intersection of both boxes are assigned only to the region exhibiting the larger overlap area, while the smaller region receives remaining items. This guarantees no duplicated lines appear in the final markdown output.

Related Components

Several files work together to support this functionality:

• src/extractor/content_stream.rs – Low-level TextItem extraction used by the main function
• tests/integration_tests.rs – Test suite exercising region extraction, including overlapping cases
• src/markdown/ – Downstream formatting modules that process the extracted region text

Summary

• Exclusive assignment ensures each text item belongs to exactly one region, eliminating duplication previously seen in ~21% of benchmark PDFs
• Largest overlap area serves as the deterministic criterion for resolving competing region claims on text items
• The had_candidates array tracks regions that touched items to prevent false OCR triggers when content is intentionally lost to neighbors
• The implementation resides in src/lib.rs (lines 78‑149) with supporting logic in src/extractor/content_stream.rs

Frequently Asked Questions

How does extract_text_in_regions_mem decide which region gets a text item when multiple regions overlap?

The function calculates the overlap area between the text item and each intersecting region using region_item_overlap_area. The item is assigned exclusively to the region with the greatest spatial overlap. This deterministic approach ensures consistent results across extractions while preventing duplicate content in the output.

What is the purpose of the had_candidates array in the overlap resolution algorithm?

The had_candidates array tracks which regions were touched by any text item during the assignment iteration, even if those items were ultimately assigned to a different region with larger overlap. This flag prevents the system from triggering unnecessary OCR on regions that appear empty only because their content was claimed by overlapping neighbors.

Why doesn't the function trigger OCR for regions that lost all their text to overlapping neighbors?

When a region ends up empty due to the lost_to_neighbor condition, the code checks if had_candidates is true for that region. If the region previously overlapped items but lost them to a larger neighbor, the empty result is intentional rather than a quality failure. Therefore, the function sets needs_ocr to false, avoiding redundant OCR processing.

Where is the core overlap detection logic implemented in the firecrawl/pdf-inspector codebase?

The primary logic resides in src/lib.rs within the extract_text_in_regions_mem function, specifically lines 78‑149. Helper functions for overlap detection and area calculation are implemented in the same file, while the underlying text extraction pipeline is defined in src/extractor/content_stream.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}