# Using MarkdownProfile (Fidelity vs Compact) for Token-Efficient Output in pdf-inspector

> Discover how pdf-inspector's Fidelity and Compact MarkdownProfile options reduce tokens efficiently. Preserve exact layout or achieve aggressive token reduction for optimal output.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-08-10

---

**The pdf-inspector library provides two built-in Markdown profiles—Fidelity for exact layout preservation and Compact for aggressive token reduction—controlled via the `MarkdownProfile` enum at the CLI or programmatically.**

pdf-inspector converts PDF documents to Markdown with a configurable trade-off between visual accuracy and token economy. The `MarkdownProfile` enum in [`src/markdown/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/mod.rs) defines this behavior, allowing developers to optimize output for either human readability or LLM consumption.

## What Is MarkdownProfile?

`MarkdownProfile` is a Rust enum that determines how aggressively the post-processing pipeline normalizes whitespace and cosmetic formatting:

```rust
pub enum MarkdownProfile { Fidelity, Compact }

```

- **Fidelity**: Preserves exact line breaks, spaces, and dot-leader characters (e.g., "…..")
- **Compact**: Collapses multiple spaces, removes dot-leaders, and trims trailing whitespace

This enum is declared at line 913 of [`src/markdown/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/mod.rs) and propagates through the entire extraction pipeline.

## How Each Profile Transforms Output

The profile selection affects three distinct stages:

| Stage | Fidelity Behavior | Compact Behavior |
|-------|-------------------|------------------|
| Extraction | Keeps exact spacing and dot-leaders | Normalizes whitespace, strips dot-leaders |
| Post-processing ([`src/markdown/postprocess.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/postprocess.rs)) | `fidelity_profile_preserves_dot_leaders` leaves formatting intact | `compact_profile_collapses_dot_leaders` removes/condenses decorative characters |
| Final output | Verbatim structure for tables and indentation | Streamlined text optimized for token count |

The post-processing functions are implemented starting at line 10 of [`src/markdown/postprocess.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/postprocess.rs).

## When to Use Each Profile

**Choose Fidelity when:**

- Downstream tools require precise visual structure (tables, nested lists, code blocks)
- Human readers need preserved indentation and alignment
- Token budget is not constrained

**Choose Compact when:**

- Feeding output directly to LLM prompts or embeddings
- Processing large PDFs where every token counts
- Running batch pipelines across thousands of documents

Compact mode can reduce token count by approximately 30% without sacrificing semantic content. This makes it ideal for RAG applications and automated document analysis workflows.

## CLI Usage

The `pdf2md` binary defaults to Fidelity. Enable Compact with the `--compact` flag:

```bash

# Full-fidelity output (default)

pdf2md my-document.pdf > output.md

# Compact, token-efficient output

pdf2md --compact my-document.pdf > output_compact.md

```

The flag parsing occurs at line 312 of [`src/bin/pdf2md.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/bin/pdf2md.rs), where `--compact` maps to `MarkdownProfile::Compact`.

## Programmatic Usage (Rust)

Configure the profile explicitly in `PdfOptions`:

```rust
use pdf_inspector::{process_pdf_with_options, MarkdownProfile};

let opts = pdf_inspector::PdfOptions {
    markdown: pdf_inspector::MarkdownOptions {
        profile: MarkdownProfile::Compact,
        ..Default::default()
    },
    ..Default::default()
};

let result = process_pdf_with_options("my-document.pdf", opts)?;
println!("{}", result.markdown.text);

```

## Python Binding

Via the `pyo3` interface:

```python
import pdf_inspector

opts = pdf_inspector.PdfOptions()
opts.markdown.profile = pdf_inspector.MarkdownProfile.Compact

result = pdf_inspector.process_pdf("my-document.pdf", opts)
print(result.markdown.text)

```

## Key Source Files

| File | Purpose |
|------|---------|
| [`src/markdown/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/mod.rs) | `MarkdownProfile` enum definition (line 913) |
| [`src/markdown/postprocess.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/postprocess.rs) | Profile-specific dot-leader handling (lines 10–12) |
| [`src/bin/pdf2md.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/bin/pdf2md.rs) | CLI flag parsing for `--compact` (line 312) |
| [`tests/integration_tests.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/tests/integration_tests.rs) | Verification tests for both profiles (lines 537–563) |

## Testing Profile Behavior

The integration test suite validates both profiles:

- Lines 537+: Fidelity profile produces full expected output with preserved formatting
- Lines 547+: Compact profile demonstrates measurable token reduction while maintaining semantic equivalence

These tests ensure that profile selection behaves predictably across PDF document types.

## Summary

- `MarkdownProfile` controls the fidelity-versus-efficiency trade-off in pdf-inspector
- Fidelity preserves exact visual structure; Compact minimizes tokens through aggressive normalization
- Configure via CLI `--compact` flag or `MarkdownOptions.profile` in Rust/Python
- Compact reduces token count ~30%—critical for LLM pipelines
- Core implementation spans [`mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/mod.rs), [`postprocess.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/postprocess.rs), and [`pdf2md.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/pdf2md.rs)

## Frequently Asked Questions

### How do I enable Compact mode from the command line?

Pass the `--compact` flag to `pdf2md`. The default behavior without this flag uses Fidelity mode, as implemented in [`src/bin/pdf2md.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/bin/pdf2md.rs) at line 312.

### Will Compact mode break table formatting?

Compact mode removes cosmetic whitespace and dot-leaders but preserves semantic structure. Tables remain parseable as Markdown, though visual alignment of cell content may shift. For applications requiring pixel-perfect table reproduction, use Fidelity mode.

### Can I switch profiles dynamically at runtime?

Yes. The `MarkdownProfile` is passed through `PdfOptions` to `process_pdf_with_options`, allowing runtime selection based on document type, user preference, or downstream consumer requirements.

### What exactly does Compact mode remove?

According to [`src/markdown/postprocess.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/markdown/postprocess.rs), Compact mode collapses multiple consecutive spaces, strips dot-leader sequences (repeated periods used for visual alignment), and trims trailing whitespace from lines. These changes target token-heavy formatting that carries no semantic meaning for language models.