# How ToUnicode CMap Parsing Decodes CID-Encoded Fonts (Type0 and Identity-H) in pdf-inspector

> Learn how ToUnicode CMap parsing decodes CID-encoded fonts like Type0 and Identity-H. Discover the process of converting CID fonts to Unicode in pdf-inspector.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-08-14

---

**ToUnicode CMap parsing converts CID-encoded fonts to Unicode by reading the `ToUnicode` stream, determining 1-byte or 2-byte code widths, building fallback mappings from TrueType tables, and handling CID-to-GID remapping for subsetted fonts.**

`pdf-inspector` is an open-source Rust library for extracting text from PDF documents. When working with **CID-encoded fonts**—specifically Type0 fonts using `Identity-H` or `Identity-V` encodings—the library must interpret character IDs (CIDs) rather than simple byte sequences. This article explains the complete decoding pipeline implemented in the source code.

## Overview of CID Font Decoding

CID fonts represent glyphs as **Character IDs** instead of direct character codes. The `ToUnicode` CMap attached to a PDF font provides the mapping from these CIDs to Unicode code points. The `pdf-inspector` implementation follows a six-step process to reliably extract text even from poorly-encoded or subsetted documents.

## Step 1: Parse the Raw ToUnicode Stream

The entry point `build_cmap_entry_from_stream` in [`src/tounicode.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/tounicode.rs) reads the decompressed `ToUnicode` stream and constructs a `ToUnicodeCMap` object.

```rust
// src/tounicode.rs, lines 32-44
pub fn build_cmap_entry_from_stream(stream: &[u8]) -> Option<ToUnicodeCMap> {
    ToUnicodeCMap::parse(stream).ok()
}

```

The `parse` method extracts three critical sections from the CMap:

- **`begincodespacerange`** — defines valid code ranges
- **`bfchar`** — single character mappings
- **`bfrange`** — range-based character mappings

The parser also records source hex lengths to determine the code byte width in subsequent steps.

## Step 2: Determine the Code Byte Width

CID fonts require **2-byte big-endian CIDs**, while simple fonts may use single-byte codes. After parsing, the code calculates `code_byte_length` by analyzing the ranges found in the CMap.

```rust
// src/tounicode.rs, lines 71-92
impl ToUnicodeCMap {
    fn parse(data: &[u8]) -> Result<Self, CMapError> {
        let mut cmap = ToUnicodeCMap::default();
        // ... parsing logic ...
        
        // Determine byte width from maximum code value
        cmap.code_byte_length = if max_code <= 0xFF { 1 } else { 2 };
        Ok(cmap)
    }
}

```

This distinction is essential because Type0 fonts always emit 2-byte CIDs, and misinterpreting the width corrupts the text extraction.

## Step 3: Build Fallback Mappings

When the ToUnicode CMap is missing or incomplete, `pdf-inspector` constructs fallback mappings from the embedded font data:

| Fallback Type | Function | Source Location |
|-------------|----------|---------------|
| TrueType cmap table | `build_cmap_from_truetype` | `src/tounicode.rs:67-71` |
| Simple font cmap | `build_simple_cmap_from_truetype` | Same file |

These functions extract the `cmap` table from TrueType fonts, reverse the GID→Unicode mapping, and synthesize a functional `ToUnicodeCMap`.

## Step 4: Handle CID-to-GID Mismatches in Subsetted Fonts

PDF generators often subset fonts by renumbering GIDs while preserving the original ToUnicode CMap. This creates a mismatch between CIDs in the content stream and entries in the CMap.

The `try_remap_subset_cmap` function detects this condition specifically for `Identity-H/V` fonts:

```rust
// src/tounicode.rs, lines 80-86
pub fn try_remap_subset_cmap(
    cmap: ToUnicodeCMap,
    font_dict: &Dictionary,
    doc: &Document,
    obj_number: u32,
) -> (ToUnicodeCMap, Option<ToUnicodeCMap>) {
    // Check if font encoding is Identity-H or Identity-V
    // and if CID range suggests subsetting
    if is_subsetted(&cmap) {
        // Attempt remapping using CIDToGIDMap or sequential GIDs
        return remap_to_sequential(cmap);
    }
    (cmap, None)
}

```

The remapping strategy uses either:

- **CIDToGIDMap** — if present in the font descriptor, directly maps CIDs to correct GIDs
- **Sequential remapping** — assumes GIDs were renumbered sequentially starting from 0

## Step 5: Decode CID Bytes to Unicode

The `decode_cids` function processes raw bytes from PDF content streams according to the determined `code_byte_length`:

```rust
// src/tounicode.rs, lines 50-66
impl ToUnicodeCMap {
    pub fn decode_cids(&self, bytes: &[u8]) -> String {
        let mut result = String::new();
        let mut i = 0;
        
        while i < bytes.len() {
            let (cid, consumed) = if self.code_byte_length == 2 {
                // Big-endian 2-byte CID
                let cid = ((bytes[i] as u16) << 8) | bytes[i + 1] as u16;
                (cid, 2)
            } else {
                (bytes[i] as u16, 1)
            };
            
            match self.lookup(cid) {
                Some(unicode) => result.push_str(&unicode),
                None if self.cid_passthrough => {
                    // Last resort: treat CID as Unicode code point
                    if let Some(c) = char::from_u32(cid as u32) {
                        result.push(c);
                    }
                }
                None => { /* unmapped */ }
            }
            i += consumed;
        }
        result
    }
}

```

For **Identity-H fonts**, the `cid_passthrough` flag enables direct CID-to-Unicode conversion when the CMap lookup fails. This works because Identity-H encoding uses Unicode values as CIDs.

## Step 6: Quality Check and Final Fallback

If more than half of the CIDs remain unmapped, `decode_cids` returns an empty string (lines 103-111), signaling the caller to try alternative strategies such as glyph name extraction.

## Why This Works for Type0 and Identity-H/V

- **Type0 fonts** are inherently CID-based; the encoding determines how CIDs map to glyphs
- **Identity-H/V** indicates Unicode values were used directly as CIDs, making passthrough decoding viable
- The 2-byte code width assumption aligns with the **big-endian CID** convention in PDF specifications

## Summary

- `build_cmap_entry_from_stream` initiates parsing from raw PDF streams
- `code_byte_length` detection distinguishes 1-byte simple fonts from 2-byte CID fonts
- TrueType `cmap` tables provide fallback mappings when ToUnicode is absent
- `try_remap_subset_cmap` corrects GID renumbering in subsetted Identity-H/V fonts
- `decode_cids` performs the final CID→Unicode conversion with passthrough support
- Unmapped code thresholds trigger alternative extraction strategies

## Frequently Asked Questions

### What is a ToUnicode CMap in PDF files?

A **ToUnicode CMap** is a mapping embedded in PDF fonts that converts character identifiers (CIDs or character codes) to Unicode code points. Without this mapping, text extraction tools cannot determine what Unicode characters correspond to the glyphs displayed in the PDF. According to the `pdf-inspector` source code, these CMaps contain `begincodespacerange`, `bfchar`, and `bfrange` sections that define the decoding rules.

### Why do Type0 fonts require special handling compared to simple fonts?

**Type0 fonts** are Composite fonts that use CIDs instead of direct character codes. The PDF content stream contains 2-byte big-endian CID values rather than ASCII or single-byte encodings. The `pdf-inspector` implementation detects this through `code_byte_length == 2` and processes the bytes accordingly in `decode_cids`. Simple fonts typically use 1-byte codes that map directly to glyph indices.

### What happens when a PDF has no ToUnicode CMap?

When the ToUnicode CMap is missing, `pdf-inspector` executes fallback logic in `build_cmap_from_truetype` or `build_simple_cmap_from_truetype`. These functions extract the `cmap` table from embedded TrueType font data and construct a reverse mapping from GID to Unicode. For **Identity-H/V fonts**, the code additionally enables `cid_passthrough` mode, treating CIDs directly as Unicode code points when other methods fail.

### How does pdf-inspector handle font subsetting?

Font subsetting renumbers GIDs while often preserving the original ToUnicode CMap, causing CID-to-GID misalignment. The `try_remap_subset_cmap` function detects this by checking if the minimum CID in the CMap exceeds expected values for Identity-H/V encodings. It then applies either a **CIDToGIDMap** from the font descriptor or **sequential remapping** to realign the CMap entries with the actual subsetted GIDs.