# How the pdf-inspector Layout Module Differentiates Newspaper vs. Tabular Reading Orders

> Learn how pdf-inspectors layout module identifies newspaper vs tabular reading orders. Discover its techniques for column detection, line grouping, and layout classification.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-09-02

---

**The pdf-inspector layout engine distinguishes newspaper reading order from tabular order by detecting columns, grouping lines within each column, and applying density, balance, and prose heuristics to classify the layout type.**

The **pdf-inspector** library from Firecrawl uses a sophisticated three-stage pipeline to determine whether a multi-column PDF page should be read as newspaper-style prose (columns processed sequentially) or as a table (rows interleaved across columns). This decision directly impacts how extracted text flows and how downstream table detection operates.

## Column Detection: The Foundation of Layout Analysis

The process begins with `detect_columns` in [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) (lines 21–86). This function builds a **horizontal occupancy histogram** across the page, identifying gutter valleys that separate columns. It validates these valleys using absolute or relative thresholds and returns a vector of `ColumnRegion` objects describing each detected column.

```rust
// Detect columns on a page
let columns = detect_columns(&items, page_number, page_has_table);

```

Without reliable column detection, no downstream reading order classification is possible. The algorithm handles both clean gutter-based layouts and noisier scanned documents.

## Per-Column Line Grouping and Prose Analysis

For each detected column, the engine groups items into rough text lines using Y-proximity clustering. The `columns_have_prose` helper (lines 557–573) performs critical measurements:

- **Line count** per column
- **Full-line ratio**: what portion of lines span ≥45% of column width

This data feeds directly into the newspaper-or-table decision.

## The is_newspaper_layout Decision Function

The core classification logic resides in `is_newspaper_layout`, declared at lines 1832–1834 of [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs). This function applies four interlocking heuristics:

### 1. Density Floor (15-Line Minimum)

Both columns must contain **at least 15 lines** (commented at lines 1903–1904). Sparse columns fail immediately and default to tabular treatment. This prevents fragmentary sidebars or captions from triggering false newspaper detection.

### 2. Balanced Line Counts

Columns with similar line counts (within a factor of 2) indicate **independent text flows** characteristic of newspaper layouts. Severely imbalanced counts suggest asymmetric tables or marginalia.

### 3. Prose Evidence via columns_have_prose

Each column undergoes deeper scrutiny in `columns_have_prose` (lines 669–674, 595–604, 668–670). A column qualifies as prose when it satisfies:

- **Minimum width**: ≥120 points
- **Minimum lines**: ≥8 lines
- **Full-line ratio**: ≥40% of lines cover ≥45% of column width, **OR** a consecutive run of ≥6 full lines exists
- **Item density**: average items per line ≤3.5 (prose has fewer, larger items than table cells)

### 4. Unanimous Column Agreement

**All** detected columns must satisfy the prose criteria for newspaper classification. If any column fails, the engine falls back to **tabular reading order** where rows are interleaved across columns.

```rust
// Decide reading order based on column analysis
let is_newspaper = is_newspaper_layout(&per_column_lines, &columns);
let reading_order = if is_newspaper { "newspaper" } else { "tabular" };

```

## Test Suite Validation

The implementation is validated by targeted unit tests in [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs):

- `test_newspaper_layout_detection` (lines 3000–3045): Creates two dense columns with matching Y positions, expects `true`
- `test_newspaper_layout_misaligned_baselines` (lines in test block): Confirms detection works even with independent column baselines
- Sparse column test (lines 3094–3135): Verifies the 15-line floor correctly returns `false`

These tests ensure the heuristics perform robustly across real-world document variations.

## Key Source Files

| File | Responsibility |
|------|---------------|
| [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) | Column detection, valley validation, `is_newspaper_layout`, `columns_have_prose` heuristics |
| [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) | Public API exposure and comprehensive unit tests |
| [`src/tables/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tables/mod.rs) | Consumes reading-order decisions for table processing logic |

## Summary

- **Column detection** via `detect_columns` establishes the geometric structure
- **Line grouping** within columns measures density and fill patterns
- **Density floor**: 15+ lines required per column
- **Balance check**: line counts within 2× factor
- **Prose verification**: width, full-line ratio, and item density thresholds
- **Unanimous requirement**: all columns must qualify as prose for newspaper classification
- Falls back to **tabular reading order** when any heuristic fails

## Frequently Asked Questions

### How does pdf-inspector handle documents with more than two columns?

The `is_newspaper_layout` function evaluates **all** detected columns uniformly. Every column must pass the density, balance, and prose criteria. Multi-column newspaper layouts with three or more columns follow the same unanimous voting logic—any column that fails prose detection triggers tabular fallback.

### What happens when columns have significantly different line counts?

Severely imbalanced line counts (beyond a 2× ratio) indicate the columns are **not independent text flows**. This commonly occurs in tables with header rows spanning one column, or documents with main text plus narrow sidebar comments. The engine treats such cases as tabular layouts.

### Why does the prose detection use a 3.5 items-per-line threshold?

Prose paragraphs typically contain **continuous text runs** with few discrete items per typographic line. Table cells, conversely, often contain single words, numbers, or short phrases—yielding higher item counts. The ≤3.5 threshold distinguishes these patterns empirically, as implemented in `columns_have_prose` at lines 668–670.

### Can the newspaper detection thresholds be configured?

The current implementation in [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) uses **hardcoded constants**: 15-line floor, 120pt minimum width, 45% full-line threshold, 40% full-line ratio, 6-line consecutive run, and 3.5 items-per-line maximum. These values reflect tuned heuristics for general document collections. Users seeking different behavior must modify the source constants directly.