# CID Font Support in pdf-inspector: How It Decodes Type0/Identity-H Fonts

> Learn how pdf-inspector decodes Type0/Identity-H CID fonts using a three-stage fallback: ToUnicode CMap parsing, TrueType cmap extraction, and CID-to-Unicode passthrough.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-09-02

---

**pdf-inspector handles CID-based Type0 fonts with Identity-H encoding through a three-stage fallback strategy: ToUnicode CMap parsing, embedded TrueType cmap extraction, and final CID-to-Unicode passthrough when CIDs directly encode Unicode scalar values.**

Extracting readable text from modern PDFs often fails when generators like Chromium or wkhtmltopdf strip the ToUnicode mapping from CID-keyed fonts. The `firecrawl/pdf-inspector` repository solves this with a robust decoder that recovers text even from stripped Identity-H fonts while falling back to OCR when necessary. This article examines the complete CID font support implementation across the Rust source code.

## Understanding the CID Font Problem

PDFs using **Type0 fonts with Identity-H encoding** store characters as 2-byte **Character IDs (CIDs)** rather than direct Unicode code points. The `/ToUnicode` CMap normally translates CIDs to Unicode, but many PDF generators remove this mapping to reduce file size. Without it, standard extractors produce gibberish or empty output.

pdf-inspector addresses this gap through coordinated detection, fallback construction, and intelligent passthrough decoding.

## Stage 1: Font Detection and CMap Collection

The extraction pipeline begins in [`src/extractor/fonts.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/fonts.rs) with `collect_cmaps_from_fonts`, which scans every font dictionary on a page to identify Type0 fonts using Identity-H or Identity-V encoding.

```rust
// src/extractor/fonts.rs, lines 31-44 — simplified conceptual flow
fn collect_cmaps_from_fonts(page: &Page) -> HashMap<u32, CMapEntry> {
    for font in page.fonts.values() {
        if let Some(font_dict) = font.as_dict() {
            // Check for Type0 with Identity-H/Identity-V encoding
            if is_type0_with_identity_encoding(font_dict) {
                let descendant = get_descendant_font(font_dict);
                let font_file_obj = get_font_file2_obj_num(descendant);
                // Use font_file_obj as key for fallback CMap
            }
        }
    }
}

```

The `get_font_file2_obj_num` function locates the embedded font data through the **FontDescriptor** → **FontFile2** or **FontFile3** chain. This object number becomes the key for associating fallback CMaps with specific fonts.

## Stage 2: ToUnicode Parsing with Fallback Construction

When `build_cmap_entry_from_stream` in [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs) cannot parse a valid or complete ToUnicode CMap, it constructs a **fallback CMap** containing:

- The partially parsed primary map
- An embedded TrueType cmap if available
- A built-in generic cmap as last resort

```rust
// src/tounicode.rs, lines 32-45 — fallback CMap construction
fn build_cmap_entry_from_stream(stream: Option<&Stream>) -> CMapEntry {
    match parse_tounicode_stream(stream) {
        Ok(map) if map.is_complete() => CMapEntry::primary(map),
        Ok(partial) => CMapEntry::with_fallback(partial, extract_truetype_cmap()),
        Err(_) => CMapEntry::fallback_only(built_in_cmap()),
    }
}

```

## Stage 3: CID-as-Unicode Passthrough Fallback

The critical innovation for Identity-H support appears in [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs) (lines 38-54). When no valid mapping exists, pdf-inspector analyzes the **CID font's `/W` (widths) array**:

- If CID values predominantly match Unicode code points (≥ 0x41 for ASCII, typical Unicode ranges)
- The system creates an empty `ToUnicodeCMap` with `cid_passthrough = true`

```rust
// src/tounicode.rs, lines 38-54 — CID passthrough detection
fn create_cid_passthrough_cmap(font_obj: u32, widths: &[u16]) -> ToUnicodeCMap {
    let mut cmap = ToUnicodeCMap::new();
    cmap.code_byte_length = 2;      // Identity-H always uses 2-byte CIDs
    cmap.cid_passthrough = true;    // Enable direct CID → Unicode casting
    
    // Analysis of widths array confirms Unicode-like CID distribution
    if appears_to_encode_unicode(widths) {
        cmap.register_for_font(font_obj);
        cmap
    } else {
        mark_for_ocr(font_obj)
    }
}

```

This heuristic works because many generators (notably Chromium's PDF engine) use Unicode scalar values directly as CIDs when constructing Identity-H fonts.

## Stage 4: Runtime Byte Stream Decoding

The `ToUnicodeCMap::decode_bytes` method implements the actual decoding logic in [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs) (lines 78-90). For 2-byte CID streams, it first attempts normal map lookup, then falls back to direct CID-to-Unicode conversion:

```rust
// src/tounicode.rs, lines 78-90 — the passthrough code path
impl ToUnicodeCMap {
    pub fn decode_bytes(&self, bytes: &[u8]) -> Option<String> {
        let mut result = String::new();
        for chunk in bytes.chunks(self.code_byte_length) {
            let cid = u16::from_be_bytes([chunk[0], chunk[1]]);
            
            if let Some(unicode) = self.map.get(&cid) {
                result.push(*unicode);
            } else if self.cid_passthrough {
                // Direct CID → Unicode passthrough for Identity-H fonts
                if let Some(ch) = char::from_u32(cid as u32) {
                    result.push(ch);
                }
            }
        }
        Some(result)
    }
}

```

The **critical `else` branch** enables text recovery when the ToUnicode map is missing but CIDs encode valid Unicode scalars.

## Stage 5: OCR Fallback Detection

When passthrough fails or CIDs do not correspond to Unicode, [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) (around line 543) flags the page for OCR processing:

```rust
// src/detector.rs — Identity-H detection for OCR fallback
fn has_identity_h_without_tounicode(page: &Page) -> bool {
    page.fonts.values().any(|font| {
        let is_identity = font.encoding == "Identity-H" || font.encoding == "Identity-V";
        let undecodable = !font.has_valid_tounicode && !font.cid_passthrough_applied;
        is_identity && undecodable
    })
}

```

The detection occurs in `should_use_ocr_for_page`, which combines this check with other undecodability signals.

## Practical Usage

pdf-inspector applies this entire pipeline automatically through its high-level API:

```rust
// Automatic CID font handling
let md = pdf_inspector::process_pdf_with_options("input.pdf", Default::default())?;
println!("{}", md);  // Text from Identity-H fonts correctly decoded

```

To inspect the internal CMap decisions for debugging:

```rust
use pdf_inspector::tounicode::ToUnicodeCMap;

// Manual passthrough CMap construction
let mut cmap = ToUnicodeCMap::new();
cmap.code_byte_length = 2;
cmap.cid_passthrough = true;

// Verify decoding behavior
let decoded = cmap.decode_bytes(&[0x00, 0x41]); // CID 65 → 'A'
assert_eq!(decoded, Some("A".to_string()));

```

## Summary

- **Font scanning** in [`src/extractor/fonts.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/fonts.rs) identifies Type0/Identity-H fonts and locates embedded font objects via `get_font_file2_obj_num`
- **Multi-stage fallback** in [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs) attempts ToUnicode parsing, TrueType cmap extraction, then CID-passthrough construction
- **Heuristic passthrough** enables Unicode recovery when generators use CIDs as direct Unicode values, controlled by `cid_passthrough` flag
- **Runtime decoding** in `ToUnicodeCMap::decode_bytes` applies the passthrough in the 2-byte CID loop's fallback branch
- **OCR detection** in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) marks pages as undecodable when all strategies fail, ensuring no silent data loss

## Frequently Asked Questions

### What is an Identity-H font in PDF?

An **Identity-H font** is a Type0 composite font where the encoding name `/Identity-H` indicates that CIDs map directly to horizontal glyph indices without additional CMap transformation. This encoding commonly appears in PDFs generated by web browsers. While efficient for rendering, it often lacks the `/ToUnicode` CMap needed for text extraction, which pdf-inspector addresses through heuristic passthrough decoding.

### Why does pdf-inspector use CID passthrough instead of always using OCR?

**CID passthrough preserves vector text accuracy** at native resolution, whereas OCR introduces recognition errors, layout artifacts, and slower processing. The passthrough strategy succeeds for the majority of modern PDFs where CIDs encode Unicode directly. OCR serves only as a controlled fallback when the heuristic fails, maintaining extraction quality while maximizing coverage.

### How does pdf-inspector detect when passthrough will fail?

pdf-inspector analyzes the **CID font's `/W` (widths) array** during `create_cid_passthrough_cmap`. If CID values fall outside expected Unicode ranges or show non-text distributions, the font is flagged as undecodable. The detector in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) then combines this signal with other font properties to make page-level OCR decisions.

### Can I force OCR for all Identity-H fonts?

The current API does not expose a direct "force OCR" flag for specific encodings. However, you can preprocess pages by inspecting `CMapEntry` construction results or by modifying the detector logic in a fork. The architecture separates detection ([`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs)) from extraction, making such customizations straightforward.