PDF Text Layer Extraction and Parseability Checking in ai-job-search: A Technical Deep Dive

The ai-job-search repository validates PDF parseability by extracting hidden text layers using Poppler's pdftotext, normalizing whitespace, and applying configurable sanity checks for page count, character minimums, and required phrases.

The ai-job-search project ensures generated résumés remain ATS-readable by verifying that PDFs contain extractable text rather than scanned images. This verification pipeline leverages external PDF utilities to parse document structure and validate content accessibility. Understanding how PDF text layer extraction and parseability checking is implemented helps developers maintain robust document generation workflows.

Poppler-Based PDF Processing Architecture

The verification system relies on the Poppler suite of PDF tools, specifically pdfinfo and pdftotext. In tools/verify_pdf.py, the helper function run_tool executes these external binaries and handles error states.

When run_tool invokes a command, it captures stdout and raises a custom VerificationError if the binary is missing (suggesting installation of poppler-utils) or if the process returns a non-zero exit code. This ensures clear feedback when system dependencies are unavailable.

Extracting Page Metadata and Text Content

Parsing Page Counts with pdfinfo

The parse_page_count function processes pdfinfo output using a regular expression that searches for lines beginning with Pages: followed by an integer. If the pattern is absent, the function raises VerificationError, preventing validation of malformed documents.

Normalizing Extracted Text for Robust Matching

Raw text extracted via pdftotext often contains irregular whitespace that interferes with string matching. The normalize_text utility collapses all whitespace sequences into single spaces, ensuring that subsequent checks for required phrases remain reliable regardless of formatting variations.

Core Verification Logic in verify_pdf

The main verify_pdf routine performs four independent checks to confirm PDF parseability:

File existence validation confirms the target path exists using Path(pdf_path).is_file(), raising VerificationError immediately if the file is missing.

Page count verification calls run_tool(["pdfinfo", str(pdf_path)]), parses the result through parse_page_count, and compares the value against the expected_pages parameter. This ensures the document contains the correct number of pages.

Text layer extraction invokes run_tool(["pdftotext", "-layout", str(pdf_path), "-"]) to extract the hidden text layer, then applies normalize_text and validates the length against min_chars. This check confirms the PDF contains searchable text rather than image-only content.

Required phrase validation iterates over the required_text tuple, normalizing each phrase and verifying membership in the extracted content. This guarantees essential sections like contact information or professional experience headers are present.

When any check fails, VerificationError bubbles up to the CLI, which reports a formatted error to stderr and exits with a non-zero status.

Unit Testing the Verification Pipeline

The test suite in tests/test_verify_pdf.py uses unittest.mock.patch to stub run_tool and validate behavior without requiring actual PDF files or external dependencies.

Tests verify that parse_page_count correctly interprets Pages: output lines, that successful verification passes when all criteria are met, and that proper errors surface for wrong page counts, insufficient text length, missing required phrases, and non-existent files. Additional tests confirm that missing external commands trigger helpful installation prompts.

Implementation Example

from tools.verify_pdf import verify_pdf, VerificationError

try:
    verify_pdf(
        "output/resume.pdf",
        expected_pages=2,
        min_chars=200,
        required_text=("Professional Experience", "john.doe@example.com")
    )
    print("PDF passes all checks.")
except VerificationError as err:
    print(f"Verification failed: {err}")

Command-line usage provides the same validation capabilities:

python -m tools.verify_pdf output/resume.pdf \
    --pages 2 \
    --min-chars 200 \
    --contains "Professional Experience" \
    --contains "john.doe@example.com"

Summary

• The verification pipeline depends on Poppler's pdfinfo and pdftotext binaries installed on the host system.
• All logic resides in tools/verify_pdf.py, with comprehensive test coverage in tests/test_verify_pdf.py.
• Text normalization collapses whitespace to ensure reliable phrase matching regardless of PDF formatting.
• Four distinct checks validate file existence, page count, minimum character count, and required phrase presence.
• The VerificationError exception model provides clear, actionable feedback for debugging invalid PDFs.

Frequently Asked Questions

What external dependencies are required for PDF text extraction?

The system requires the Poppler suite, specifically the pdfinfo and pdftotext binaries typically packaged as poppler-utils on Debian-based systems or poppler on macOS via Homebrew. If these commands are missing, run_tool raises a VerificationError with installation instructions.

How does the system handle PDFs that are image-based without text layers?

When pdftotext extracts content from image-only PDFs, it returns empty or minimal text. The verify_pdf function checks the normalized text length against the min_chars parameter, raising VerificationError if the content falls below the threshold, effectively flagging unscanned documents as unparsable.

Can the verification check for multiple required phrases simultaneously?

Yes, the required_text parameter accepts a tuple of strings. The validation logic in tools/verify_pdf.py iterates through each phrase, normalizing it and verifying membership in the extracted text layer. All phrases must be present for validation to pass.

Why is whitespace normalization necessary before checking text content?

PDF text extraction often preserves irregular spacing, line breaks, and formatting artifacts from the original document. The normalize_text function collapses these sequences into single spaces, ensuring that phrase matching works reliably regardless of how the PDF generator structured the hidden text layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}