How pdf-inspector Detects Reading Order: A Deep Dive into Image-Anchored Flow Logic

pdf-inspector determines reading order by building a topological region graph for each PDF page, using image-anchored cues to identify column layouts and emitting text in the correct sequence.

The firecrawl/pdf-inspector library handles complex PDF layouts—newspaper columns, hero images, and mixed-direction text—through a three-stage reading order detection system implemented in Rust. This article examines the core algorithms in src/extractor/reading_order.rs and their integration with the broader extraction pipeline.

Understanding the Three-Stage Pipeline

Reading order detection in pdf-inspector proceeds through tightly coupled stages:

Stage	Purpose	Key Function
1. Column-flow band detection	Identify image-anchored cues that define column boundaries	infer_image_anchored_flow
2. Region graph construction	Partition page into directional regions as a DAG	build_region_graph
3. RTL-aware linearization	Reorder columns for right-to-left text and emit final sequence	Internal ordering logic

If image-anchored detection fails, the system falls back to a classic page-wide histogram approach in src/detector.rs.

Stage 1: Image-Anchored Flow Detection

The entry point infer_image_anchored_flow (L82 in reading_order.rs) attempts two complementary strategies for identifying column layouts.

Full-Width Image Flow

The local_flow_below_full_width_image helper (L45) detects layouts where a prominent image anchors the start of a multi-column section:

• Finds a single image spanning ≥65% of page width with roughly square proportions
• Scans 200-400 points below for aligned rows that split cleanly (gap ≥ MIN_ROW_GUTTER)
• Clusters split positions and validates vertical gaps of 60-120 points
• Returns a ColumnFlowBand with split_x, y_bottom, and y_top on success

This handles common news article layouts where a hero image precedes body text in columns.

Paired-Column Images Flow

The paired_column_images helper (L51) handles layouts with multiple images distributed across columns:

• Requires a pre-detected split near page center (40-60% of width)
• Demands ≥3 images confined to each column side (minimum MIN_IMAGE_WIDTH × MIN_IMAGE_HEIGHT)
• Requires ≥3 "wide" images (≥35% page width) forming a vertical stack across both sides
• Validates balance with line_balance < 0.55 to reject decorative banners

Both helpers return None on heuristic failure, triggering fallback to histogram-based detection.

Stage 2: Building the Region Graph

build_region_graph (L96) transforms detected bands into traversable structure:

// Item classification (simplified from L4-L12)
let region = if item.y > band.y_top {
    RegionKind::Above
} else if item.y < band.y_bottom {
    RegionKind::Below
} else if item.x + item.width / 2.0 < band.split_x {
    RegionKind::Left
} else {
    RegionKind::Right
};

The function then:

1. Partitions items into four buckets: above, left column, right column, below
2. Detects RTL text via is_rtl_text (L14 in text_utils.rs) across column contents
3. Orders nodes (L16-22): Above → Column(s) → Below, with columns reversed for RTL
4. Emits RegionNode vector containing kind and associated items (L24-27)

The resulting DAG enforces dependencies: above-content precedes columns, which precede below-content.

Stage 3: RTL-Aware Ordering

Right-to-left detection occurs early in build_region_graph. When RTL content is found in either column:

• Column order reverses: right column first, then left
• Linearization follows: FullWidth(Above) → Column(Right) → Column(Left) → FullWidth(Below)

This ensures Arabic, Hebrew, and Persian documents maintain proper reading direction within their structural regions.

Integration with the Layout Engine

The layout orchestrator in src/extractor/layout.rs wires reading order into full extraction:

// From group_into_lines_with_thresholds_and_regions_impl (≈L27-L40)
if let Some(band) = image_regions.get(&page).and_then(|regions| {
    super::reading_order::infer_image_anchored_flow(
        &page_items,
        regions,
        detected_split,
    )
}) {
    for node in super::reading_order::build_region_graph(page_items, band) {
        all_lines.extend(group_single_column(node.items, adaptive_threshold));
    }
    continue; // Skip histogram fallback
}

When no band is detected, execution falls through to detect_columns in src/detector.rs.

Direct API Usage Examples

Example 1: Manual Reading Order Extraction

use pdf_inspector::extractor::reading_order::{
    infer_image_anchored_flow, build_region_graph,
};
use pdf_inspector::types::TextItem;

// items: Vec<TextItem> for a single page
// images: detected bounding boxes as (x0, y0, x1, y1)
let split = None; // No pre-detected split

if let Some(band) = infer_image_anchored_flow(&items, &images, split) {
    let graph = build_region_graph(items, band);
    for node in graph {
        println!("Processing {:?} with {} items", node.kind, node.items.len());
    }
}

Example 2: High-Level Layout API

use pdf_inspector::extractor::layout::group_into_lines_with_thresholds_and_regions;

let lines = group_into_lines_with_thresholds_and_regions(
    all_items,        // Vec<TextItem> across document
    &page_thresholds, // HashMap<PageNum, f32>
    &table_pages,     // HashSet<PageNum>
    &chart_regions,   // HashMap<PageNum, Vec<Rect>>
    &image_regions,   // HashMap<PageNum, Vec<Rect>>
);

This automatically applies image-anchored reading order detection where applicable.

Key Source Files

File	Role
src/extractor/reading_order.rs	Core detection (infer_image_anchored_flow, build_region_graph)
src/extractor/layout.rs	Pipeline integration and line grouping
src/detector.rs	Fallback histogram-based column detection
src/types.rs	TextItem, RegionKind, and data structures
src/text_utils.rs	RTL detection and width utilities

Summary

• Image-anchored flow detection uses hero images or paired column images to identify split positions and vertical ranges
• Region graph construction partitions pages into above/columns/below as a DAG
• RTL awareness reverses column order when right-to-left text is detected
• Automatic fallback to histogram detection when image heuristics fail
• Zero configuration—the library selects the best strategy per-page

Frequently Asked Questions

How does pdf-inspector handle documents with both left-to-right and right-to-left text?

The is_rtl_text function in src/text_utils.rs scans column contents during build_region_graph. If either column contains RTL text, their order reverses (right-first, then left). This occurs at lines L16-22 of reading_order.rs, ensuring mixed-direction documents maintain correct reading flow.

What happens when no images are present on a page?

The infer_image_anchored_flow function returns None, causing the layout engine in layout.rs to skip the region graph path and fall back to detect_columns in src/detector.rs. This histogram-based approach analyzes text alignment patterns without image cues.

Can the detection thresholds be customized for specific document types?

The current implementation uses compile-time constants (MIN_ROW_GUTTER, MIN_IMAGE_WIDTH, MIN_IMAGE_HEIGHT) and hardcoded percentages (65% width for hero images, 35% for wide images). These are not exposed as configuration parameters in the public API. Forking or patching src/extractor/reading_order.rs would be required for domain-specific adjustments.

How does the system distinguish genuine two-column layouts from decorative image arrangements?

The paired_column_images function enforces multiple validation rules: minimum image counts per side, "wide" image requirements for vertical stack detection, and the line_balance < 0.55 ratio check. These constraints reject banner-like image placements that lack the structural properties of readable columns.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}