# How pdf-inspector Handles PDF Operator States: A Deep Dive into the Rust State Machine

> Explore how pdf-inspector manages PDF operator states with a Rust state machine. Learn about stack operations and matrix transformations in this deep dive.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-08-31

---

**pdf-inspector tracks PDF operator states using a minimal Rust state machine in [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) that mirrors the PDF graphics and text model, with stack-based save/restore operations via `q`/`Q` operators and matrix transformations via `cm`.**

The [firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector) crate parses PDF content streams to extract structured text while preserving layout. Understanding how it handles **PDF operator states** reveals why it can accurately position glyphs, detect underlines, and respect nested transformations. This article breaks down the state machine implementation, from graphics stack management to text matrix calculations.

## The Core State Machine Architecture

pdf-inspector's extractor follows a classic interpreter pattern: iterate over decoded operations, update mutable state, and emit `TextItem` objects when text is rendered. The implementation lives primarily in [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs), with roughly 200 lines dedicated to operator dispatch.

The state machine manages several distinct but interacting subsystems:

- **Graphics state** — transformation matrices, line width, clipping paths
- **Text state** — current font, size, spacing, rendering mode
- **Text positioning** — text matrix (Tm) and line matrix (Tlm)
- **Marked content** — accessibility tags and logical structure

Each subsystem responds to specific PDF operators, and their interaction determines where each glyph appears in the final output.

## Graphics State: The q/Q Stack Pattern

PDF graphics state uses a **stack-based save/restore mechanism**. The `q` operator pushes the current state; `Q` pops and restores it.

In [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) (lines 39–53), this is implemented with a `Vec<SavedGraphicsState>`:

```rust
if op.operator == "q" {
    gstate_stack.push(SavedGraphicsState {
        ctm,
        line_width,
        text_rendering_mode,
        // ... other fields
    });
} else if op.operator == "Q" {
    if let Some(saved) = gstate_stack.pop() {
        ctm = saved.ctm;
        line_width = saved.line_width;
        text_rendering_mode = saved.text_rendering_mode;
        // ... restore all fields
    }
}

```

This pattern is **critical for correctness**: PDF generators frequently wrap transformations in `q`/`Q` pairs to isolate changes. Without proper stack handling, a rotation or scale applied to one element would leak into subsequent elements.

The `SavedGraphicsState` struct captures a complete snapshot, ensuring that any graphics state change—no matter how deep in the nesting—is properly scoped.

## Current Transformation Matrix (CTM) Handling

The **CTM** maps user space coordinates to device space. It's a 6-element affine transformation matrix that every coordinate passes through.

The `cm` operator concatenates a new matrix onto the existing CTM. In [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs) (lines 67–79):

```rust
let new_matrix = [
    op.operands[0].as_f64_or_0(),
    op.operands[1].as_f64_or_0(),
    op.operands[2].as_f64_or_0(),
    op.operands[3].as_f64_or_0(),
    op.operands[4].as_f64_or_0(),
    op.operands[5].as_f64_or_0(),
];
ctm = multiply_matrices(&new_matrix, &ctm);

```

Note the **matrix multiplication order**: `new_matrix * ctm`, not `ctm * new_matrix`. This follows the PDF specification where new transformations apply "after" existing ones in the pipeline.

The `multiply_matrices` helper performs standard 3×3 matrix multiplication on the 2D affine representation. Incorrect order here would cause translations, rotations, and scales to compose backwards—producing misplaced text in documents with complex transformations.

## Line Width and Stroke State

For detecting **underlines** and other decorative lines, pdf-inspector tracks the current line width via the `w` operator:

```rust
"w" => {
    line_width = op.operands[0].as_f64_or_0();
}

```

This simple assignment (lines 81–85 in [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs)) enables downstream layout analysis. When the extractor later encounters a rectangle or line-drawing operator, it can compare the stroke width against font metrics to identify underlines versus borders.

## Text Block Management with BT/ET

PDF text operations must occur inside **text objects** bounded by `BT` (begin text) and `ET` (end text) operators. The state machine tracks this with a boolean flag and resets positioning state on entry:

```rust
"BT" => {
    in_text_block = true;
    text_matrix = [1.0, 0.0, 0.0, 1.0, 0.0, 0.0];
    line_matrix = [1.0, 0.0, 0.0, 1.0, 0.0, 0.0];
}
"ET" => {
    in_text_block = false;
}

```

This reset behavior (lines 86–96) is **spec-mandated**: each text object starts with identity matrices, preventing text positioning from leaking across unrelated blocks.

## Text State: Font, Size, and Rendering Mode

Inside a text block, the **text state** controls how glyphs are rendered. Key operators handled in [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs) (lines 98–124):

| Operator | Purpose | Variable Updated |
|----------|---------|----------------|
| `Tf` | Set font and size | `current_font`, `font_size` |
| `TL` | Set text leading | `text_leading` |
| `Tr` | Set rendering mode | `text_rendering_mode` |
| `Tc` | Set character spacing | `char_spacing` |
| `Tw` | Set word spacing | `word_spacing` |
| `Ts` | Set text rise | `rise` |

The **rendering mode** (`Tr`) deserves special attention. Mode 3 means "neither fill nor stroke"—invisible text often used for OCR layer alignment. pdf-inspector checks this value when emitting `TextItem`s, either skipping invisible glyphs or recording them for retry logic when no visible text is found.

## Text Positioning: Tm, Tlm, and Glyph Placement

The heart of accurate text extraction lies in **text matrix management**. PDF uses two matrices:

- **Text matrix (Tm)** — defines the coordinate system for the current glyph sequence
- **Line matrix (Tlm)** — remembers where the current line started, used for line-breaking operators

Operators that modify positioning (lines 124–170):

- `Td tx ty` — move to start of next line, offset by (tx, ty) from line matrix start
- `TD tx ty` — same as `Td` but also sets text leading to ty
- `Tm a b c d e f` — absolutely set the text matrix (used for precise positioning)
- `T*` — move to next line using current text leading

After each **show-text** operator (`Tj`, `TJ`, `'`, `"`), the matrix advances by the computed string width. This auto-advancement is what makes PDF text extraction challenging: you must measure each string, account for spacing parameters, and update the matrix before the next operation.

## Show-Text Operators: From Bytes to TextItems

The final stage converts PDF string operands into positioned `TextItem` structures. In [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs) (lines 75–115), the show-text handler:

1. Decodes the operand bytes using the current font's encoding
2. Computes glyph widths via `compute_string_width_ts`
3. Adjusts for text rise (`rise_adjusted` translation)
4. Transforms through CTM to get device-space coordinates
5. Creates a `TextItem` with bounds, content, and font metadata

Invisible text (rendering mode 3) receives special handling: the extractor may skip it or flag it for accessibility processing, depending on configuration.

## Marked Content and Logical Structure

PDF accessibility relies on **marked content** operators that tag regions with MCIDs (marked content IDs) and `ActualText` for alternative representations.

The state machine maintains a `marked_content_stack` (lines 124–136):

```rust
"BMC" | "BDC" => {
    let tag = op.operands[0].as_name_str().unwrap_or_default();
    let mcid = /* extract from properties */;
    marked_content_stack.push(MarkedContentEntry { tag: tag.to_string(), mcid });
}
"EMC" => {
    marked_content_stack.pop();
}

```

Each emitted `TextItem` captures the innermost MCID, enabling later reconstruction of document structure for screen readers and semantic analysis.

## XObject Integration: Images and Forms

The `Do` operator draws **XObjects**—external objects including images and form XObjects (nested content streams). pdf-inspector handles these in [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs) (lines 165–210) and [`src/extractor/xobjects.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/xobjects.rs).

For **images**, it computes a bounding box from the current CTM and emits a synthetic `TextItem`:

```rust
let image_bbox = image_bbox_from_ctm(&ctm);
items.push(TextItem::ImagePlaceholder {
    rect: image_bbox,
    // ...
});

```

For **form XObjects**, it recursively calls `extract_form_xobject_text`, passing the **current CTM as the inherited transformation**. This nesting is essential: form XObjects have their own content streams but inherit and concatenate with the parent's coordinate system.

The recursion is bounded by a `FormWalkBudget` to prevent infinite loops from malformed PDFs with circular form references.

## Practical API Usage

The state machine is exercised through two primary interfaces. For standard extraction:

```rust
use pdf_inspector::{process_pdf_with_options, ProcessOptions};

fn main() -> Result<(), pdf_inspector::PdfError> {
    let opts = ProcessOptions {
        json: true,
        ..Default::default()
    };
    let (json, _warnings) = process_pdf_with_options("sample.pdf", &opts)?;
    println!("{}", json);
    Ok(())
}

```

For custom pipelines that need direct state machine access:

```rust
use pdf_inspector::{extractor::content_stream::extract_page_text_items, FontCMaps};

fn custom_extract(doc: &lopdf::Document, page_id: lopdf::ObjectId) {
    let font_cmaps = FontCMaps::default();
    let mut style_cache = pdf_inspector::extractor::fonts::FontStyleCache::new();
    let mut form_budget = pdf_inspector::extractor::xobjects::FormWalkBudget::default();

    let (page_extraction, _, _, _) = extract_page_text_items(
        doc,
        page_id,
        1,
        &font_cmaps,
        false,
        &mut style_cache,
        &mut form_budget,
    ).unwrap();

    for item in page_extraction.0 {
        println!("{:?}", item);
    }
}

```

Both paths ultimately invoke the operator state machinery in [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs), ensuring consistent handling of graphics state, text positioning, and nested structures.

## Key Files and Responsibilities

| File | Role in PDF Operator State Handling |
|------|-------------------------------------|
| [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) | Core state machine; implements operator dispatch, matrix math, stack management, and `TextItem` emission |
| [`src/extractor/fonts.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/fonts.rs) | Font encoding tables and glyph width resolution; consumed by show-text operators |
| [`src/extractor/xobjects.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/xobjects.rs) | XObject lookup and recursive form processing; manages `FormWalkBudget` |
| [`src/extractor/mod.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/mod.rs) | Pipeline orchestration; calls page extraction repeatedly |
| [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) | PDF type detection; gates whether operator state machine runs |

## Summary

- pdf-inspector handles **PDF operator states** through a Rust state machine in [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) that faithfully implements the PDF graphics and text models
- **Graphics state** uses a `Vec<SavedGraphicsState>` stack manipulated by `q` (push) and `Q` (pop) operators, ensuringscoped transformations
- The **CTM** is updated via matrix multiplication in the `cm` operator, with correct order (`new * existing`) per the PDF specification
- **Text state** variables—font, size, spacing, rendering mode—are set by dedicated operators and consumed during show-text processing
- **Text positioning** relies on text matrix (Tm) and line matrix (Tlm) updates via `Td`, `TD`, `Tm`, and `T*` operators, with automatic advancement after each string
- **Marked content** and **XObjects** extend the state machine with stack-based tagging and recursive form processing, both respecting inherited transformations

## Frequently Asked Questions

### How does pdf-inspector handle nested transformations in PDFs?

pdf-inspector uses the `q` (save) and `Q` (restore) operators to manage a stack of `SavedGraphicsState` structs. When `q` is encountered, the current CTM and other graphics state are cloned and pushed; `Q` pops and restores them. This stack-based approach correctly scopes transformations regardless of nesting depth, as implemented in [`src/extractor/content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/extractor/content_stream.rs) lines 39–53.

### What determines the final position of extracted text?

Final text position results from cascading transformations: the **CTM** (set by `cm` and inherited from parent contexts) multiplied by the **text matrix** (set by `Tm` or updated by `Td`/`TD`/`T*`), plus **text rise** adjustment. The show-text operators apply this combined transformation to each glyph's origin, producing device-space coordinates stored in `TextItem.x` and `TextItem.y`.

### How does pdf-inspector detect and handle invisible text?

The `Tr` (set rendering mode) operator sets `text_rendering_mode`. Mode 3 means "invisible" (neither fill nor stroke). The show-text handler in [`content_stream.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/content_stream.rs) checks this value and either skips invisible glyphs or records them for retry logic, depending on whether the document contains visible alternatives. This handles common OCR-invisible text patterns.

### Can pdf-inspector extract text from PDF forms and annotations?

Yes, via the **`Do` operator** handler. When a form XObject is referenced, `extract_form_xobject_text` is called recursively with the current CTM as the inherited transformation. This processes the nested content stream while preserving positioning context. The recursion is bounded by `FormWalkBudget` to prevent infinite loops from malformed circular references.