# Garbage Text Upgrade in Firecrawl PDF‑Inspector: How Mixed PDFs Become Scanned PDFs

> Learn about Firecrawl PDF-Inspector's garbage text upgrade. Discover how it converts Mixed PDFs to Scanned PDFs using OCR when alphanumeric text is below 50% for better data extraction.

- Repository: [Firecrawl/pdf-inspector](https://github.com/firecrawl/pdf-inspector)
- Tags: deep-dive
- Published: 2026-08-14

---

**The garbage text upgrade is a post‑extraction safeguard that reclassifies a Mixed PDF as Scanned when its extracted Markdown contains fewer than 50% alphanumeric characters, triggering full‑document OCR to replace unreadable content.**

The `firecrawl/pdf‑inspector` library uses a two‑phase classification system to determine whether a PDF needs optical character recognition (OCR). After initial detection labels a document as **Mixed** (containing both text operators and images), the library re‑examines the actual extracted content. If that content is mostly non‑alphanumeric "garbage," the garbage text upgrade promotes the PDF to **Scanned** status—ensuring OCR runs on every page.

---

## How PDF Classification Works in pdf‑inspector

The classification pipeline begins with page sampling and continues through post‑extraction validation.

### Phase 1: Initial Detection (Mixed PDF Identification)

In [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) (lines 81–130), the library samples pages to determine the PDF type:

- **Text operators found** → potential `PdfType::Text` or `PdfType::Mixed`
- **Images present** with text → `PdfType::Mixed`
- **Images only** → `PdfType::Scanned`

A Mixed PDF initially suggests that some selectable text exists alongside raster or vector images. However, this text may be corrupted, CID‑mapped gibberish, or otherwise unusable.

### Phase 2: Garbage Text Validation

After Markdown extraction, the library validates content quality. This is where the garbage text upgrade applies.

---

## The Garbage Text Upgrade Mechanism

The upgrade logic resides in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) (lines 4150–4185). The code checks whether a Mixed PDF's extracted Markdown meets quality thresholds before finalizing its type.

### Step‑by‑Step Upgrade Process

1. **Extract Markdown** — The library processes the PDF and generates Markdown output
2. **Evaluate quality** — Call `is_garbage_text()` from [`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs) (lines 428–447)
3. **Apply threshold** — If non‑whitespace characters are predominantly non‑alphanumeric, return `true`
4. **Reclassify** — When `pdf_type == PdfType::Mixed && is_garbage_text(...)` is satisfied, upgrade to `PdfType::Scanned`

```rust
// src/lib.rs (simplified logic structure)
if pdf_type == PdfType::Mixed && is_garbage_text(&extracted_markdown) {
    // Garbage text upgrade triggered
    pdf_type = PdfType::Scanned;
    ocr_recommended = true;
    pages_needing_ocr = all_pages; // Every page queued for OCR
}

```

### What Defines "Garbage" Text?

The `is_garbage_text` function in [`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs) analyzes character distribution:

- **Counts non‑whitespace characters**
- **Categorizes as alphanumeric versus symbols/punctuation**
- **Returns `true` when alphanumeric ratio falls below 50%**

This "symbol soup" detection catches common PDF corruption patterns:

- CID‑encoded garbage from missing font mappings
- Binary data interpreted as text operators
- Encoding errors producing Unicode replacement characters

---

## Practical Impact: When OCR Becomes Mandatory

The garbage text upgrade ensures downstream consumers receive usable content. Without it, a Mixed PDF with corrupted embedded text would pass through without OCR—delivering useless output.

### Code Example: Observing the Upgrade Path

```rust
use pdf_inspector::{detect_pdf_type, process_pdf_with_options, PdfType};

// Initial detection samples pages and identifies Mixed type
let detection = detect_pdf_type("damaged_report.pdf")?;
assert_eq!(detection.pdf_type, PdfType::Mixed);

// Full extraction and quality evaluation occurs here
let result = process_pdf_with_options("damaged_report.pdf", Default::default())?;

// If garbage text upgrade activated:
// - result.pdf_type now reflects PdfType::Scanned
// - result.ocr_recommended == true
// - result.pages_needing_ocr contains all page numbers

```

### Key Consequences of Upgrade

| Aspect | Before Upgrade | After Upgrade |
|--------|---------------|---------------|
| **PDF Type** | `Mixed` | `Scanned` |
| **OCR Flag** | `false` (selective) | `true` (mandatory) |
| **Pages Processed** | Text pages skipped | All pages OCR'd |
| **Output Quality** | Potentially garbled Markdown | Clean, searchable text |

---

## Related Quality Checks in text_quality.rs

The [`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs) module provides additional validation functions used alongside `is_garbage_text`:

- **`is_cid_garbage`** — Detects CID‑specific corruption patterns
- **Character frequency analysis** — Identifies encoding failures
- **Heuristic scoring** — Combines multiple signals for robust detection

These utilities support the garbage text upgrade by providing precise, context‑aware quality metrics rather than simple regex matching.

---

## Summary

The garbage text upgrade in `firecrawl/pdf‑inspector` is a critical quality safeguard:

- **Initial detection** in [`src/detector.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/detector.rs) may label corrupted documents as `Mixed` based on structural signals
- **`is_garbage_text`** in [`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs) evaluates actual content quality using alphanumeric density
- **Reclassification** in [`src/lib.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/lib.rs) promotes Mixed PDFs to Scanned when content falls below 50% alphanumeric
- **Full OCR** is triggered automatically, replacing garbage output with legible, searchable text

This mechanism prevents the library from emitting unusable "symbol soup" and ensures consistent, high‑quality Markdown extraction across all PDF types.

---

## Frequently Asked Questions

### What triggers the garbage text upgrade specifically?

The upgrade triggers when two conditions align: the PDF is classified as `Mixed` during initial detection, and `is_garbage_text()` returns `true` for the extracted Markdown. This occurs when fewer than 50% of non‑whitespace characters are alphanumeric—indicating the embedded text is corrupted or CID‑mapped garbage rather than meaningful content.

### How does pdf‑inspector detect garbage text algorithmically?

The `is_garbage_text` function in [`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs) (lines 428–447) performs character‑level analysis. It iterates through non‑whitespace characters, categorizing each as alphanumeric or symbol/punctuation, then compares the ratio. A document with predominantly symbols, box‑drawing characters, or replacement glyphs scores as garbage regardless of total character count.

### Why promote to Scanned rather than re‑extract with different settings?

Promoting to `PdfType::Scanned` leverages the existing OCR pipeline rather than implementing parallel recovery logic. This design choice centralizes text reliability in OCR engines—which handle rasterized text robustly—rather than attempting to repair fundamentally broken PDF text operators or font encodings.

### Can the alphanumeric threshold be configured?

Based on the current source code in [`src/text_quality.rs`](https://github.com/firecrawl/pdf-inspector/blob/main/src/text_quality.rs), the 50% alphanumeric threshold appears to be a compiled heuristic rather than a user‑configurable parameter. The library prioritizes correct classification over customization, using hardcoded thresholds that have proven effective across diverse PDF corpora.