# How to Debug PDF Extraction Layout with RUST_LOG in firecrawl/pdf-inspector

> Debug PDF extraction layout with RUST_LOG in firecrawl/pdf-inspector. Set RUST_LOG to trace column detection, validation failures, and fallback paths in real-time via stderr.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: debugging-guide
- Published: 2026-09-01

---

**Set the `RUST_LOG` environment variable to `pdf_inspector::extractor::layout=debug` to emit diagnostic messages from the layout detection pipeline to stderr, allowing you to trace column detection, validation failures, and fallback paths in real-time.**

The firecrawl/pdf-inspector repository uses the standard Rust `log` crate for structured diagnostics throughout its PDF-to-Markdown extraction pipeline. When dealing with complex multi-column documents, debugging the layout detection logic is essential for resolving reading-order errors or misidentified column boundaries. By configuring the `RUST_LOG` environment variable, you can inspect the internal state of the histogram-based column detection algorithm without modifying source code.

## Core Layout Detection Functions

The layout detection logic resides in **[`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs)**. This module implements a projection-profile algorithm that identifies column gutters by analyzing horizontal whitespace valleys. Three primary functions orchestrate the detection process, each emitting structured debug logs when `RUST_LOG` is configured appropriately.

### The `detect_columns` Entry Point

The **`detect_columns`** function (lines 21–38) serves as the primary entry point for column analysis. It builds a horizontal projection histogram of text items, identifies valleys (gutters) in the distribution, and constructs `ColumnRegion` structures. When debugging is enabled, this function logs the page number and item count at initialization, allowing you to verify that the correct `TextItem` set is entering the layout engine.

### The `try_xy_cut_split` Fallback

When primary valley detection fails, the algorithm falls back to **`try_xy_cut_split`** (lines 138–166). This function searches for the largest horizontal gap between items and attempts to split the page into two columns. Debug output from this function includes the gap coordinate (`x=…`), gap width in points, and the resulting left/right item distributions, helping you identify why the primary detection missed a column boundary.

### The `validate_and_build_columns` Validator

The **`validate_and_build_columns`** function (lines 225–260) filters candidate valleys against heuristics. It checks vertical overlap between potential columns, excludes list-marker artifacts, and enforces minimum item thresholds (`MIN_ITEMS_MAJOR` and `MIN_ITEMS_MINOR`). Debug messages indicate whether valleys passed validation or if all candidates were rejected, triggering additional fallback mechanisms.

## Interpreting RUST_LOG Debug Output

When you set `RUST_LOG=pdf_inspector::extractor::layout=debug`, the library streams diagnostic messages to **stderr**. These logs trace the execution flow through the detection pipeline:

- `page X: detect_columns: N items` — Indicates the start of analysis for a specific page and the cardinality of the input set.
- `page X: XY-cut split at x=… (gap=…pt, left=…, right=…)` — Emitted by `try_xy_cut_split` when the fallback path activates.
- `page X: relative valley detection found N columns` — Logs results from alternative valley detection strategies.
- `page X: … valleys found but none passed validation` — Indicates `validate_and_build_columns` rejected all candidates due to heuristic failures.
- `page X: … columns detected (boundaries: …)` — Confirms successful detection with final boundary coordinates.

## Debugging Specific Layout Issues

Tracing these log messages allows you to diagnose five critical aspects of the extraction process:

**Item Collection Verification.** Confirm that the expected `TextItem` objects are reaching `detect_columns`. If the item count differs from the visible text blocks on the page, the issue likely resides in the upstream content stream parser rather than the layout module.

**Histogram Threshold Analysis.** The algorithm uses constants like `NOISE_FRACTION` and `MIN_GUTTER_WIDTH` to filter histogram noise. Debug output reveals the calculated thresholds versus actual valley depths, helping you determine if whitespace gaps are being incorrectly classified as noise.

**Valley Geometry Inspection.** Review the gutter positions relative to page margins. The debug logs show whether detected valleys meet the minimum distance requirements from page edges and whether they align with visual column boundaries.

**Validation Heuristic Evaluation.** When columns are detected but rejected, the logs indicate which `MIN_ITEMS` constraint or vertical overlap check failed. This is particularly useful for documents with narrow sidebars or table-of-contents columns that might not meet the major/minor item ratio requirements.

**Fallback Path Tracing.** The logs reveal whether the algorithm utilized the XY-cut method or relative valley detection after primary detection failed, helping you understand why certain layouts trigger alternative parsing strategies.

## Command Examples for Debugging

Use these shell commands to enable diagnostics in different scenarios:

```bash

# Enable layout debugging only, suppressing normal output

RUST_LOG=pdf_inspector::extractor::layout=debug \
cargo run --bin pdf2md -- example.pdf > /dev/null

```

```bash

# Debug layout alongside table detection for documents with tabular columns

RUST_LOG=pdf_inspector::extractor::layout=debug,pdf_inspector::tables=debug \
cargo run --bin pdf2md -- example.pdf > /dev/null

```

```bash

# Capture debug output to a file for offline analysis

RUST_LOG=pdf_inspector::extractor::layout=debug \
cargo run --bin pdf2md -- example.pdf 2> layout.log

```

```bash

# Correlate raw PDF operators with layout decisions

RUST_LOG=pdf_inspector::extractor::content_stream=trace,pdf_inspector::extractor::layout=debug \
cargo run --bin pdf2md -- example.pdf > /dev/null

```

**Command Component Breakdown:**

- `RUST_LOG=pdf_inspector::extractor::layout=debug` — Targets debug-level logs from the layout module specifically.
- `> /dev/null` — Discards the generated Markdown output (stdout), leaving only diagnostic logs (stderr) visible.
- `2> layout.log` — Redirects stderr to a file, preserving the debug stream for analysis in text editors or grep pipelines.
- `content_stream=trace` — Enables fine-grained tracing of PDF operator parsing in [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs), useful when layout issues stem from incorrect text item generation.

## Related Debugging Modules

While [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) handles column detection, several adjacent modules influence layout behavior and provide complementary debug information:

**[`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs)** — Controls the low-level PDF operator parser. When layout detection appears to miss text blocks, enable `pdf_inspector::extractor::content_stream=trace` to verify that text positioning operators are being interpreted correctly before they reach the layout engine.

**[`src/tables/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tables/mod.rs)** — Implements table detection logic that executes after column identification. If columns contain embedded tables, debugging both modules simultaneously (`layout=debug,tables=debug`) reveals whether table structures are interfering with column boundary detection.

**[`docs/debugging.md`](https://github.com/firecrawl/pdf-inspector/blob/main/docs/debugging.md)** — Contains the official reference for all available `RUST_LOG` targets and verbosity levels within the repository.

**[`src/markdown/analysis.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/analysis.rs)** — Logs Y-gap and paragraph-threshold analysis that follows layout detection. Use this module's debug output to verify that correctly detected columns are being rendered with proper reading order in the final Markdown output.

## Summary

- The firecrawl/pdf-inspector library uses the Rust `log` crate to emit diagnostic information when the `RUST_LOG` environment variable is set.
- Layout-specific debugging requires `RUST_LOG=pdf_inspector::extractor::layout=debug` to trace the `detect_columns`, `try_xy_cut_split`, and `validate_and_build_columns` functions in [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs).
- Debug output streams to stderr and includes histogram thresholds, valley coordinates, validation results, and fallback activation signals.
- Combine layout debugging with `content_stream=trace` or `tables=debug` to diagnose interactions between text parsing, column detection, and table extraction.
- Redirect stderr to log files (`2> layout.log`) for offline analysis of complex multi-page documents.

## Frequently Asked Questions

### What environment variable controls logging in pdf-inspector?

The repository follows the Rust ecosystem standard by respecting the **`RUST_LOG`** environment variable. This variable accepts module paths and log levels (e.g., `debug`, `trace`, `info`) to filter diagnostic output from specific components like `pdf_inspector::extractor::layout`.

### How do I capture debug output to a file instead of the terminal?

Since debug logs write to stderr, redirect stream 2 to a file using shell redirection. Execute your command with `2> layout.log` appended to save all `RUST_LOG` output to `layout.log` while keeping the normal program output (Markdown text) in the terminal or redirecting it elsewhere.

### Why am I not seeing any layout debug messages even with RUST_LOG set?

Verify that you are targeting the correct module path. The layout module specifically requires `pdf_inspector::extractor::layout=debug` (note the double colon separators and the underscore in `pdf_inspector`). Additionally, ensure you are not suppressing stderr with shell redirection, and confirm that the binary was compiled without `--release` flags that might strip debug symbols or log statements in some configurations.

### How can I correlate low-level PDF operators with layout decisions?

Enable tracing for both the content stream parser and the layout module simultaneously using **`RUST_LOG=pdf_inspector::extractor::content_stream=trace,pdf_inspector::extractor::layout=debug`**. This configuration prints the raw PDF operators (from [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs)) alongside the layout detection logs, allowing you to map specific text positioning commands to the resulting column boundaries detected in [`layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/layout.rs).