How to Handle Different PDF Formats and Encoding Issues

The hiring-agent repository handles diverse PDF formats and encoding issues by converting documents to Markdown using PyMuPDF's to_markdown function, which detects OCR content, normalizes Unicode, extracts tables, and filters invisible text and background colors before LLM processing.

Extracting structured data from PDFs requires robust handling of varied formats and character encodings. The interviewstreet/hiring-agent repository solves this by implementing a comprehensive PDF-to-Markdown pipeline that processes everything from searchable text documents to scanned image PDFs. This guide explains how to handle different PDF formats and encoding issues using the repository's PDFHandler class and underlying rendering engine.

The PDF Processing Pipeline

The architecture centers on two core components in the main/ directory. The PDFHandler class in main/pdf.py orchestrates the end-to-end flow, while the to_markdown function in main/pymupdf_rag.py handles the heavy-duty rendering.

PDFHandler.__init__ creates a TemplateManager (located in main/prompts/template_manager.py) and initializes the LLM provider. When processing begins, extract_text_from_pdf opens the PDF using pymupdf.open(pdf_path) and delegates rendering to to_markdown ([lines 52-57 in pdf.py](https://github.com/interviewstreet/hiring-agent/blob/main/main/pdf.py#L52-L57)).

The to_markdown function iterates page-by-page via get_page_output and builds a Markdown representation that normalizes character encodings, removes invisible text, and flags OCR-only pages. The resulting clean Markdown is then fed to section-specific LLM prompts and converted into a typed JSONResume model defined in main/models.py.

Detecting and Handling PDF Format Variations

The repository automatically categorizes incoming PDFs and applies appropriate extraction strategies.

Pure Text PDFs

For searchable text documents, the IdentifyHeaders class analyzes font sizes to distinguish body text from headings. The renderer extracts text directly from page.get_text("dict"), preserving Unicode characters without manual re-encoding. Headers are converted to Markdown syntax (#, ##, etc.) to maintain document structure.

Scanned and OCR-Only PDFs

The page_is_ocr function ([lines 1000-1010 in pymupdf_rag.py](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py#L1000-L1010)) scans the page's internal bboxlog for "ignore-text" objects. If all text blocks are flagged as ignore-text, the page is identified as image-only.

When force_text=True (the default), the renderer rasterizes the page and runs the PyMuPDF-4-LLM OCR pipeline via write_text, yielding searchable text even from image-only PDFs.

Documents with Backgrounds and Colors

The get_bg_color function ([lines 1414-1451 in pymupdf_rag.py](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py#L1414-L1451)) samples the four corners of each page. If all corners share the same RGB value, that color is stored as bg_color. During rendering, vector graphics matching this background color are ignored, preventing large colored blocks from polluting the Markdown output.

Mixed Text and Image Content

During get_page_output ([lines 1086-1132 in pymupdf_rag.py](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py#L1086-L1132)), the system collects image rectangles using page.get_image_info() and filters them by relative size using the image_size_limit parameter (default 0.05). Surviving images are rendered, and if force_text=True, the system recursively extracts any hidden text layers within image regions.

Tables and Structured Data

The renderer discovers grid structures using page.find_tables(..., strategy=table_strategy) ([lines 1191-1199 in pymupdf_rag.py](https://github.com/interviewstreet/hiring-agent/blob/main/main/pymupdf_rag.py#L1191-L1199)). Tables smaller than 2 rows or columns are dropped. Valid tables are converted to Markdown using each table's to_markdown method, maintaining logical reading order alongside paragraph text.

Unicode and Font Encoding Quirks

The renderer processes raw text spans (span["text"]) without attempting manual re-encoding, relying on PyMuPDF's native Unicode string handling. Stray control characters are stripped via s["text"].strip(). For glyphs that cannot be rendered, setting use_glyphs=True replaces missing characters with numeric placeholders, preventing Unicode errors.

Configuration Options for Encoding Resilience

The to_markdown function exposes several parameters to tune extraction behavior:

• ignore_alpha (default: True) — Skips invisible (alpha-0) text to prevent garbage from OCR artifacts. Set to False only if you specifically require invisible text layers.
• detect_bg_color (default: True) — Enables background-color detection to filter out matching vector graphics. Disable for faster processing when backgrounds are irrelevant.
• force_text (default: True) — Forces rasterization of graphics and subsequent OCR extraction. Essential for handling scanned documents but adds processing overhead.
• image_size_limit (default: 0.05) — Sets the minimum relative size for images to be considered. Adjust downward to capture small logos or upward to ignore decorative elements.
• ignore_images / ignore_graphics (default: False) — Toggle these to True for pure text extraction when speed is prioritized over fidelity.

Modify these parameters in pdf.py or when calling to_markdown directly:

from main.pymupdf_rag import to_markdown
import pymupdf

doc = pymupdf.open("resume.pdf")
resume_text = to_markdown(
    doc,
    pages=[0, 1],
    force_text=True,                # Always rasterize images for OCR

    ignore_alpha=False,             # Keep invisible text (some OCR tools use it)

    detect_bg_color=False,          # Skip background detection for speed

    image_size_limit=0.02,           # Capture smaller images

)

Extending Support for Exotic PDF Encodings

For PDFs containing custom CMap or non-standard Unicode mappings, implement a post-processing step in extract_text_from_pdf. Since the function returns a raw string, you can sanitize control characters before LLM ingestion:

import re
from main.pdf import PDFHandler

class SanitizedPDFHandler(PDFHandler):
    def extract_text_from_pdf(self, pdf_path: str) -> Optional[str]:
        # Get raw markdown from parent implementation

        raw_text = super().extract_text_from_pdf(pdf_path)
        if raw_text is None:
            return None
        # Strip control characters that might break JSON parsing

        return re.sub(r'[\x00-\x1F\x7F]', '', raw_text)

This regex removes null bytes and control characters (hex 00-1F and 7F) that could interfere with downstream processing.

Practical Implementation Example

Typical usage involves instantiating PDFHandler and extracting structured JSON:

from main.pdf import PDFHandler

handler = PDFHandler()

# Extract full résumé as typed JSONResume

json_resume = handler.extract_json_from_pdf("examples/resume.pdf")

# Access structured data via Pydantic models

print(json_resume.basics.name)          # → "Jane Doe"

print(json_resume.work[0].position)    # → "Software Engineer"

# Access raw Markdown for debugging

raw_md = handler.extract_text_from_pdf("examples/resume.pdf")
print(raw_md[:500])                     # First 500 characters of Markdown

All extraction logic resides in:

• main/pdf.py — High-level orchestration and LLM integration
• main/pymupdf_rag.py — Robust PDF-to-Markdown rendering with OCR and table support
• main/prompts/template_manager.py — Section-specific LLM prompts
• main/models.py — Pydantic schemas (JSONResume, Basics, Work, etc.)

Summary

• PDFHandler in main/pdf.py coordinates the conversion from PDF to structured JSON via Markdown.
• PyMuPDF handles the underlying document parsing, while to_markdown in main/pymupdf_rag.py manages complex layouts including tables, images, and OCR content.
• ** Automatic detection** distinguishes between searchable text, scanned images, and mixed-content PDFs using page_is_ocr and background color sampling.
• Configuration flags like force_text, ignore_alpha, and detect_bg_color allow fine-tuning for specific document types and encoding issues.
• Post-processing with regex sanitization handles exotic encodings and control characters that PyMuPDF might preserve.

Frequently Asked Questions

How does the repository detect if a PDF contains only scanned images?

The page_is_ocr function examines the page's internal bboxlog for "ignore-text" objects. If all text blocks on a page are classified as ignore-text, the page is flagged as OCR-only, triggering rasterization and text extraction via the force_text pipeline.

What configuration should I use for PDFs with encoding errors or missing glyphs?

Set force_text=True to ensure OCR fallback for unrenderable content, and enable use_glyphs=True in the to_markdown call to replace missing glyphs with numeric placeholders rather than throwing Unicode errors. Additionally, apply a post-processing regex to strip control characters ([\x00-\x1F\x7F]) from the final Markdown string.

How are complex tables extracted from multi-column PDF layouts?

The renderer uses page.find_tables() with configurable strategies to detect grid structures. Tables with fewer than 2 rows or columns are filtered out. Valid tables are converted to Markdown using their native to_markdown methods, preserving logical reading order relative to surrounding text blocks.

Can the hiring-agent process password-protected PDFs?

The current implementation in main/pdf.py uses pymupdf.open(pdf_path) without password parameters. To handle encrypted PDFs, you would need to modify extract_text_from_pdf to pass the password to pymupdf.open(pdf_path, password="your_password") or prompt for credentials before processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}