# How to Debug PDF Classification with RUST_LOG in pdf-inspector

> Debug PDF classification in pdf-inspector by setting RUST_LOG. Get structured debug output to stderr, preserving normal program output.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: how-to-guide
- Published: 2026-09-01

---

**Set the `RUST_LOG` environment variable to enable structured debug output for any module in the `pdf-inspector` Rust codebase, with logs written to stderr to preserve normal program output.**

The `pdf-inspector` repository from Firecrawl uses the `log` crate combined with `env_logger` to provide deep visibility into its PDF classification pipeline. When you need to understand why a page was classified as TextBased, Scanned, ImageBased, or Mixed, structured logging lets you inspect the intermediate values driving every decision.

## How RUST_LOG Works in pdf-inspector

The logging system follows the standard `env_logger` pattern: `module_path=level`. Each major component contains `log::debug!` and `log::trace!` statements that report operator counts, font metadata, layout heuristics, and detection thresholds.

Key modules and their paths:

- `pdf_inspector::detector` — core classification logic in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)
- `pdf_inspector::extractor::layout` — column detection and reading-order logic in [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs)
- `pdf_inspector::tables` — table detection strategies in [`src/tables/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tables/mod.rs)
- `pdf_inspector::markdown::analysis` — paragraph threshold calculations in [`src/markdown/analysis.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/analysis.rs)
- `pdf_inspector::extractor::content_stream` — raw content-stream operator analysis
- `pdf_inspector::extractor::fonts` — font metadata and CMap parsing

Logs emit to stderr, so piping stdout to `/dev/null` or a file keeps your Markdown or JSON output clean.

## Common RUST_LOG Patterns for PDF Classification Debugging

### Inspect the PDF Type Detector

To see why a document was classified as scanned or mixed:

```bash
RUST_LOG=pdf_inspector::detector=debug cargo run --release --bin detect-pdf -- file.pdf

```

This activates debug statements in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) that print per-page analysis values.

### Debug Content-Stream Operators

For the lowest-level view of operator counts that determine if a page has extractable text:

```bash
RUST_LOG=pdf_inspector::extractor::content_stream=trace cargo run --bin pdf2md -- file.pdf > /dev/null

```

Use `trace` level for maximum verbosity on text operations.

### Analyze Font and Encoding Issues

When characters decode incorrectly or ligatures fail:

```bash
RUST_LOG=pdf_inspector::extractor::fonts=debug cargo run --bin pdf2md -- file.pdf > /dev/null

```

This reveals CMap parsing decisions and ToUnicode mapping problems in the font extraction pipeline.

### Examine Column and Reading-Order Logic

For newspaper-style layouts or multi-column documents:

```bash
RUST_LOG=pdf_inspector::extractor::layout=debug cargo run --bin pdf2md -- file.pdf > /dev/null

```

The [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) module logs histogram valleys and derived column counts.

### Trace Table Detection

To follow the three detection strategies (rect-based, line-based, heuristic):

```bash
RUST_LOG=pdf_inspector::tables=debug cargo run --bin pdf2md -- file.pdf > /dev/null

```

### Monitor Paragraph Threshold Analysis

For debugging Y-gap heuristics that segment text into paragraphs:

```bash
RUST_LOG=pdf_inspector::markdown::analysis=debug cargo run --bin pdf2md -- file.pdf > /dev/null

```

### Enable All Debug Output

For comprehensive tracing across all modules:

```bash
RUST_LOG=pdf_inspector=debug cargo run --bin pdf2md -- file.pdf > /dev/null

```

### Combine Multiple Modules

Use comma-separated filters for focused cross-module debugging:

```bash
RUST_LOG=pdf_inspector::detector=debug,pdf_inspector::tables=debug cargo run --bin pdf2md -- file.pdf

```

## Interpreting Debug Output

When detector logging is active, you'll see structured entries like:

```text
DEBUG pdf_inspector::detector::detect_from_document: page 3: text_ops=12 images=1 image_count=1 template=0 unique_chars=27 alphanum=5 path_ops=0 vector_text=false image_area=0 identity_h_no_tounicode=false type3_only=false font_changes=3 decodable_fonts=true

```

These fields map directly to the `PageAnalysis` struct in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs). Use them to answer specific classification questions:

- **Why scanned?** — Check `text_ops` below threshold, `images` >= 1, and low `alphanum` count
- **Why mixed requiring OCR?** — Look for `has_template_image` or `has_vector_text` flags
- **Why wrong column count?** — Check `pdf_inspector::extractor::layout` logs for valley detection and early-exit conditions

## Filtering Output with Grep

Pipe stderr to narrow results:

```bash
RUST_LOG=pdf_inspector::detector=debug cargo run --release --bin detect-pdf -- file.pdf 2>&1 | grep "pages with text"

```

This pattern isolates specific metrics without drowning in trace-level noise.

## Key Source Files for Debugging

| File | Module Path | Purpose |
|------|-------------|---------|
| [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) | `pdf_inspector::detector` | PDF type detection and OCR recommendations |
| [`src/extractor/layout.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/layout.rs) | `pdf_inspector::extractor::layout` | Column detection and reading-order heuristics |
| [`src/tables/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tables/mod.rs) | `pdf_inspector::tables` | Table detection pipeline |
| [`src/markdown/analysis.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/analysis.rs) | `pdf_inspector::markdown::analysis` | Paragraph-gap analysis |
| [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs) | `pdf_inspector::tounicode` | CMap and ToUnicode parsing logs |

For quick reference, see [`docs/debugging.md`](https://github.com/firecrawl/pdf-inspector/blob/main/docs/debugging.md) and [`AGENTS.md`](https://github.com/firecrawl/pdf-inspector/blob/main/AGENTS.md) in the repository root.

## Summary

- **Set `RUST_LOG`** with module path and level to activate specific debug output
- **Use `debug`** for operational metrics, **`trace`** for exhaustive operator-level detail
- **Pipe stdout** to isolate logs when generating Markdown or JSON output
- **Read field names** in log output against `PageAnalysis` in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) to trace classification decisions
- **Combine modules** with comma-separated filters for cross-cutting investigations

## Frequently Asked Questions

### What log levels does pdf-inspector support?

The `env_logger` backend recognizes `error`, `warn`, `info`, `debug`, and `trace`. For PDF classification debugging, use `debug` for summary metrics and `trace` for raw operator counts. The default level is `error` when `RUST_LOG` is unset.

### How do I debug why a page was incorrectly marked as scanned?

Set `RUST_LOG=pdf_inspector::detector=debug` and examine the `text_ops`, `images`, and `alphanum` fields. A page becomes Scanned when `text_ops` falls below `min_text_ops_per_page` (default 3) while containing images with insufficient alphanumeric characters.

### Can I log to a file instead of stderr?

Redirect stderr in your shell: `RUST_LOG=pdf_inspector=debug cargo run ... 2> debug.log`. The `env_logger` configuration in `pdf-inspector` does not include file appenders, so shell redirection is the standard approach.

### Where are the threshold constants defined?

Detection thresholds live in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) as const values within the detection logic. Debug output shows the computed values compared against these thresholds, letting you identify which constraint triggered a classification without reading source code.