# PDF Text Layer Extraction and Parseability Checking in ai-job-search: A Technical Deep Dive

> Learn how ai-job-search extracts PDF text layers with pdftotext and checks parseability through configurable sanity checks for robust job data validation.

- Repository: [Mads Lorentzen/ai-job-search](https://github.com/MadsLorentzen/ai-job-search)
- Tags: deep-dive
- Published: 2026-07-26

---

**The ai-job-search repository validates PDF parseability by extracting hidden text layers using Poppler's `pdftotext`, normalizing whitespace, and applying configurable sanity checks for page count, character minimums, and required phrases.**

The ai-job-search project ensures generated résumés remain ATS-readable by verifying that PDFs contain extractable text rather than scanned images. This verification pipeline leverages external PDF utilities to parse document structure and validate content accessibility. Understanding how PDF text layer extraction and parseability checking is implemented helps developers maintain robust document generation workflows.

## Poppler-Based PDF Processing Architecture

The verification system relies on the Poppler suite of PDF tools, specifically `pdfinfo` and `pdftotext`. In [`tools/verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/verify_pdf.py), the helper function `run_tool` executes these external binaries and handles error states.

When `run_tool` invokes a command, it captures stdout and raises a custom `VerificationError` if the binary is missing (suggesting installation of poppler-utils) or if the process returns a non-zero exit code. This ensures clear feedback when system dependencies are unavailable.

## Extracting Page Metadata and Text Content

### Parsing Page Counts with pdfinfo

The `parse_page_count` function processes `pdfinfo` output using a regular expression that searches for lines beginning with `Pages:` followed by an integer. If the pattern is absent, the function raises `VerificationError`, preventing validation of malformed documents.

### Normalizing Extracted Text for Robust Matching

Raw text extracted via `pdftotext` often contains irregular whitespace that interferes with string matching. The `normalize_text` utility collapses all whitespace sequences into single spaces, ensuring that subsequent checks for required phrases remain reliable regardless of formatting variations.

## Core Verification Logic in verify_pdf

The main `verify_pdf` routine performs four independent checks to confirm PDF parseability:

**File existence validation** confirms the target path exists using `Path(pdf_path).is_file()`, raising `VerificationError` immediately if the file is missing.

**Page count verification** calls `run_tool(["pdfinfo", str(pdf_path)])`, parses the result through `parse_page_count`, and compares the value against the `expected_pages` parameter. This ensures the document contains the correct number of pages.

**Text layer extraction** invokes `run_tool(["pdftotext", "-layout", str(pdf_path), "-"])` to extract the hidden text layer, then applies `normalize_text` and validates the length against `min_chars`. This check confirms the PDF contains searchable text rather than image-only content.

**Required phrase validation** iterates over the `required_text` tuple, normalizing each phrase and verifying membership in the extracted content. This guarantees essential sections like contact information or professional experience headers are present.

When any check fails, `VerificationError` bubbles up to the CLI, which reports a formatted error to stderr and exits with a non-zero status.

## Unit Testing the Verification Pipeline

The test suite in [`tests/test_verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tests/test_verify_pdf.py) uses `unittest.mock.patch` to stub `run_tool` and validate behavior without requiring actual PDF files or external dependencies.

Tests verify that `parse_page_count` correctly interprets `Pages:` output lines, that successful verification passes when all criteria are met, and that proper errors surface for wrong page counts, insufficient text length, missing required phrases, and non-existent files. Additional tests confirm that missing external commands trigger helpful installation prompts.

## Implementation Example

```python
from tools.verify_pdf import verify_pdf, VerificationError

try:
    verify_pdf(
        "output/resume.pdf",
        expected_pages=2,
        min_chars=200,
        required_text=("Professional Experience", "john.doe@example.com")
    )
    print("PDF passes all checks.")
except VerificationError as err:
    print(f"Verification failed: {err}")

```

Command-line usage provides the same validation capabilities:

```bash
python -m tools.verify_pdf output/resume.pdf \
    --pages 2 \
    --min-chars 200 \
    --contains "Professional Experience" \
    --contains "john.doe@example.com"

```

## Summary

- The verification pipeline depends on Poppler's `pdfinfo` and `pdftotext` binaries installed on the host system.
- All logic resides in [`tools/verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/verify_pdf.py), with comprehensive test coverage in [`tests/test_verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tests/test_verify_pdf.py).
- Text normalization collapses whitespace to ensure reliable phrase matching regardless of PDF formatting.
- Four distinct checks validate file existence, page count, minimum character count, and required phrase presence.
- The `VerificationError` exception model provides clear, actionable feedback for debugging invalid PDFs.

## Frequently Asked Questions

### What external dependencies are required for PDF text extraction?

The system requires the Poppler suite, specifically the `pdfinfo` and `pdftotext` binaries typically packaged as `poppler-utils` on Debian-based systems or `poppler` on macOS via Homebrew. If these commands are missing, `run_tool` raises a `VerificationError` with installation instructions.

### How does the system handle PDFs that are image-based without text layers?

When `pdftotext` extracts content from image-only PDFs, it returns empty or minimal text. The `verify_pdf` function checks the normalized text length against the `min_chars` parameter, raising `VerificationError` if the content falls below the threshold, effectively flagging unscanned documents as unparsable.

### Can the verification check for multiple required phrases simultaneously?

Yes, the `required_text` parameter accepts a tuple of strings. The validation logic in [`tools/verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/verify_pdf.py) iterates through each phrase, normalizing it and verifying membership in the extracted text layer. All phrases must be present for validation to pass.

### Why is whitespace normalization necessary before checking text content?

PDF text extraction often preserves irregular spacing, line breaks, and formatting artifacts from the original document. The `normalize_text` function collapses these sequences into single spaces, ensuring that phrase matching works reliably regardless of how the PDF generator structured the hidden text layer.