# How pdf-inspector's Font Encoding Fallback Mechanism Decodes PDFs Without ToUnicode CMaps

> Discover how pdf-inspector's font encoding fallback mechanism decodes PDFs without ToUnicode CMaps using multiple strategies for robust text extraction.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: internals
- Published: 2026-09-01

---

**pdf-inspector extracts text by first attempting to parse the embedded ToUnicode CMap, then falling back through encoding-based, TrueType cmap table, and simple-font strategies when that CMap is missing, corrupted, or too sparse.**

The `firecrawl/pdf-inspector` library implements a robust, multi-layered fallback mechanism for **font encoding in PDF text extraction**. When a PDF lacks a usable *ToUnicode* CMap—the standard mapping from character codes to Unicode—pdf-inspector reconstructs character maps from the font's intrinsic encoding data. This fallback system is fully implemented in **[`src/tournicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tournicode.rs)** and ensures reliable text recovery even from poorly encoded documents.

## How the Primary ToUnicode Parsing Works

The extraction process begins with the embedded *ToUnicode* CMap. This is the standard PDF mechanism for mapping internal character codes to Unicode code points.

```rust
if let Some(cmap) = ToUnicodeCMap::parse(data) { … }

```

When this parsing succeeds, the resulting map becomes the primary character map for decoding. However, pdf-inspector applies a **sparsity threshold** to determine if the primary map is actually usable.

### The Sparse-Map Threshold

If the parsed CMap contains fewer than **10 entries**, it is considered too sparse to provide meaningful character coverage:

```rust
if primary_entries < 10 { … }

```

This threshold triggers the fallback chain described below.

## The Three-Tier Fallback Chain in pdf-inspector

When the primary *ToUnicode* CMap fails or is too sparse, pdf-inspector executes three independent fallback generators in sequence. Each returns an `Option<ToUnicodeCMap>`:

### 1. Encoding-Based Fallback

The first fallback builds a character map from the font's **built-in encoding** such as WinAnsi or MacRoman:

```rust
.or_else(|| build_fallback_tounicode_from_encoding(font_dict, doc))

```

This strategy works for standard PDF fonts that declare their encoding explicitly but lack a *ToUnicode* CMap.

### 2. Type-0 (TrueType) Fallback

For **CID-based Type-0 fonts**, pdf-inspector parses the embedded TrueType **cmap table** to reconstruct the character map:

```rust
.or_else(|| build_fallback_cmap_for_type0(font_dict, doc))

```

This fallback is particularly effective for modern PDFs embedding TrueType or OpenType fonts with rich Unicode coverage in their `cmap` tables.

### 3. Simple-Font Fallback

The final fallback handles **simple (non-CID) fonts** by using their embedded font cmap or glyph-name table:

```rust
.or_else(|| build_fallback_cmap_for_simple(font_dict, doc))

```

These three calls appear contiguously at **lines 40–43 of [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs)** in the pdf-inspector source code.

## Fallback Promotion: Selecting the Best Character Map

After a fallback is selected, pdf-inspector applies a **promotion logic** that compares map completeness. A more comprehensive fallback—such as a TrueType cmap with 45 entries—can replace a sparse primary map with only 3 entries:

```rust
// promotion logic shown at lines 61-73 of src/tounicode.rs

```

The most expressive mapping becomes the **primary map**, while the less complete one is retained as a secondary fallback. This ensures optimal text decoding quality.

### Final Fallback for Completely Missing CMaps

If the primary *ToUnicode* CMap cannot be parsed at all, pdf-inspector bypasses sparsity checks and directly executes the Type-0 or simple-font strategies (**lines 83–90**). The resulting map serves as the sole source of character decoding.

## Practical Example: Triggering Fallback Behavior

You can observe the fallback mechanism in action using pdf-inspector's public API. The **`process_pdf_with_options`** function in **[`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs)** orchestrates the entire extraction pipeline:

```rust
use pdf_inspector::process_pdf_with_options;
use pdf_inspector::process_mode::ProcessMode;

fn main() {
    // Load a PDF that has no ToUnicode CMap.
    let pdf_bytes = std::fs::read("samples/no_tounicode.pdf").unwrap();

    // Process with default options (fallbacks are enabled).
    let result = process_pdf_with_options(pdf_bytes, ProcessMode::default());

    println!("{}", result.markdown);
}

```

When processing `no_tounicode.pdf`, pdf-inspector:

1. Attempts *ToUnicode* CMap parsing → failure
2. Executes the fallback chain (encoding → Type-0 → simple)
3. Produces structured markdown with correctly decoded characters

## Debugging Fallback Selection

Enable debug logging to trace which fallback strategy was applied:

```bash
RUST_LOG=pdf_inspector::tounicode=debug cargo run --bin pdf2md -- samples/no_tounicode.pdf

```

Expected log output:

```

ToUnicode CMap obj=5 too sparse (3 entries); using fallback
ToUnicode CMap obj=5: TrueType fallback (45 entries) > primary (3); promoting over sequential remap

```

These messages correspond to **lines 46–53 and 61–73** of [`src/tournicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tournicode.rs) in pdf-inspector.

## Key Source Files in pdf-inspector

| File | Purpose |
|------|---------|
| [`src/tournicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tournicode.rs) | Core **ToUnicode parsing** and **fallback generation** implementation |
| [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) | Public API (`process_pdf_with_options`) that triggers CMap logic |
| [`src/extractor/fonts.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/fonts.rs) | Font-specific utilities supporting fallback strategies |
| [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) | PDF type detection that may trigger OCR fallback |

## Summary

- **Primary strategy**: Parse embedded *ToUnicode* CMap; reject if < 10 entries
- **Fallback chain**: Encoding-based → TrueType cmap table → simple-font glyph tables
- **Promotion logic**: More complete fallbacks overwrite sparse primary maps
- **API entry point**: `process_pdf_with_options` in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs)
- **Debug visibility**: `RUST_LOG=pdf_inspector::tounicode=debug` exposes fallback decisions

## Frequently Asked Questions

### What triggers the font encoding fallback in pdf-inspector?

The fallback activates when the embedded *ToUnicode* CMap is **missing**, **corrupt**, or contains **fewer than 10 entries**. This sparsity threshold at line 46 of [`src/tournicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tournicode.rs) ensures that incomplete maps don't produce garbled text.

### Does pdf-inspector support Chinese, Japanese, or Korean PDFs without ToUnicode CMaps?

**Yes**, through the **Type-0 fallback** (`build_fallback_cmap_for_type0`). This strategy parses the TrueType **cmap table** embedded in CID-based fonts, which often contains comprehensive Unicode mappings for CJK characters independent of the PDF's *ToUnicode* data.

### Can developers customize or disable the fallback mechanism?

The public API (`process_pdf_with_options`) does not expose direct fallback toggles. However, fallback behavior is **always enabled** in the current implementation. To modify fallback logic, you would need to fork and edit the source in **[`src/tournicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tournicode.rs)**—specifically the chain at lines 40–43.

### How does pdf-inspector's fallback differ from other PDF extraction tools?

Many tools either fail entirely or produce raw character codes when *ToUnicode* is absent. pdf-inspector's **three-tier progressive fallback** extracts usable text by progressively leveraging richer font data sources—encoding dictionaries, TrueType tables, and glyph names—which increases recovery rates for legacy or poorly generated PDFs.