How the PDF Handler Manages Tables and Formatting During Data Extraction

The PDFHandler class extracts tables and formatting from PDF resumes by delegating to the PyMuPDF-RAG to_markdown utility, which converts tables into GitHub-flavored markdown and preserves text styling through font-bitmask analysis before passing structured content to downstream LLM prompts.

The interviewstreet/hiring-agent repository processes candidate resumes by first transforming PDF documents into structured markdown. This conversion must accurately capture tabular data—such as skills matrices or employment histories—and preserve text formatting like bold headers and italicized roles to ensure reliable downstream JSON extraction by language models.

Table Detection and Markdown Conversion

The table extraction logic resides in pymupdf_rag.py within the to_markdown function. When processing a PDF page, the system employs a multi-stage pipeline to identify, filter, and serialize tabular content.

Locating Tables with find_tables

The handler invokes page.find_tables with a user-configurable table_strategy parameter that defaults to lines_strict【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L1891-L1894】. This strategy instructs PyMuPDF to analyze page geometry and detect cell boundaries based on drawn lines or whitespace patterns.

Filtering Invalid Tables and Calculating Bounding Boxes

After detection, the code filters out malformed or insignificant tables. Any table containing fewer than 2 rows or 2 columns is discarded to avoid extracting layout artifacts or single-cell headers as data tables【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L1894-L1896】. For each valid table, the system calculates a combined bounding rectangle covering both header and body regions, storing these coordinates in the tab_rects list【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L2012-L2015】.

Spatial Ordering and Markdown Emission

During text block iteration, the output_tables helper compares the vertical coordinates of tables against text rectangles. When a table's bottom edge (tab_rect.y1) is positioned at or above a text block's top edge (lrect.y0), the system writes the table's markdown representation before processing that text block【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L4649-L4663】. This ensures tables appear in the output stream according to their visual page order. After all text blocks are processed, any remaining tables are appended at the document end【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L4945-L4950】.

Each Table object provides a to_markdown method that serializes cells into GitHub-flavored markdown using pipe-separated columns (|), preserving the original alignment and structure.

# Example: Table output from a professional experience section

"""

## Professional Experience

| Company   | Role              | Dates               |
|-----------|-------------------|---------------------|
| Acme Inc. | Senior Engineer   | Jan 2020 – Present  |
| Beta Ltd. | Software Intern   | Jun 2019 – Dec 2019 |
"""

Formatting Preservation Through Font Analysis

Beyond tables, the write_text function in pymupdf_rag.py analyzes PDF font descriptors to reconstruct markdown styling. This ensures that emphasis and structural cues from the original resume translate into tokens that LLMs can reliably interpret.

Header Detection via IdentifyHeaders

The IdentifyHeaders class analyzes font sizes across the document to establish a baseline hierarchy. When write_text processes a line, it queries this header detector; if the line qualifies as a heading, the function prefixes it with the appropriate number of # characters (H1 through H6) based on font metrics【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L822-L830】.

Inline Style Mapping via Font Flags

For every text span, the code inspects bitmask flags to determine styling:

• Bold: Applied when flags & 16 is true or char_flags & 8 is set, wrapping text with **【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L843-L858】
• Italic: Triggered by flags & 2, wrapping text with _ (underscores)
• Monospace/Code: Activated by flags & 8, converting content to fenced code blocks when the line is fully monospaced and IGNORE_CODE is False【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L800-L808】
• Strikethrough: Detected via char_flags & 1, rendered with ~~ tildes

Lists and Hyperlink Resolution

The parser recognizes bullet characters (-, *, or Unicode bullet glyphs) and prefixes the corresponding markdown line with - to generate proper unordered lists【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L860-L872】. Additionally, the resolve_links function scans spans for embedded hyperlinks, converting them to standard markdown syntax [text](url)【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L332-L393】.

# Example: Preserved formatting in markdown output

"""
**John Doe**  
_Software Engineer_  
Located in [San Francisco](https://maps.example.com/sf)

Key skills:
- Python
- Machine Learning
"""

Integration with the Resume Parsing Pipeline

The end-to-end flow begins in pdf.py where PDFHandler.extract_text_from_pdf opens the document and invokes to_markdown at the page or document level【/cache/repos/github.com/interviewstreet/hiring-agent/main/pdf.py#L47-L61】. This produces a single markdown string containing both tabular and styled text content. The handler then passes this markdown to specialized LLM prompts (defined in prompts/template_manager.py) for section-wise extraction—basics, work history, education, and skills—leveraging the preserved structure to improve JSON schema adherence in models.py.

Summary

• The PDFHandler delegates heavy lifting to PyMuPDF-RAG's to_markdown in pymupdf_rag.py, which uses page.find_tables with a default lines_strict strategy to detect tabular regions.
• Tables smaller than 2×2 are filtered out, while valid tables are positioned in the output stream based on spatial coordinates (tab_rect.y1 <= lrect.y0) before being converted to pipe-delimited markdown.
• Text formatting is preserved through bitmask analysis: bold (flags & 16), italic (flags & 2), code (flags & 8), and strikethrough (char_flags & 1), with headers identified via font-size heuristics.
• Hyperlinks convert to [text](url) syntax, and bullet lists normalize to - prefixes, ensuring the final markdown feeds reliably into downstream LLM extraction prompts.

Frequently Asked Questions

How does the hiring-agent PDF handler distinguish between real tables and layout artifacts?

The handler applies a dimensional filter in pymupdf_rag.py that requires tables to contain at least 2 rows and 2 columns before extraction【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L1894-L1896】. Single-cell or single-row regions are discarded as layout artifacts, ensuring only structured data tables proceed to markdown conversion.

What markdown syntax does the PDF handler use for extracted tables?

The system generates GitHub-flavored markdown tables using pipe-separated columns (|) and header separators (|---------|). Each Table object implements a to_markdown method that aligns cell content according to the original PDF geometry, producing valid markdown that renders correctly in LLM contexts.

How does the handler preserve bold and italic formatting from the original PDF?

The write_text function inspects font flags on each text span. Bold is detected when bit 16 is set in flags or bit 8 in char_flags, while italic is identified by bit 2 in flags【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L843-L858】. These flags trigger wrapping with ** or _ markdown markers respectively.

Can the table extraction strategy be customized when calling the PDF handler?

Yes. While PDFHandler.extract_text_from_pdf provides a high-level interface, the underlying to_markdown function accepts a table_strategy parameter that defaults to lines_strict but can be adjusted to alternative PyMuPDF strategies (such as lines or text) depending on the PDF structure【/cache/repos/github.com/interviewstreet/hiring-agent/main/pymupdf_rag.py#L1891-L1894】.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}