How the Hiring Agent Identifies and Marks Headings During PDF Text Extraction

The Hiring Agent uses font-size heuristics implemented in pymupdf_rag.py to detect headings via the max_header_id function, then prefixes qualifying lines with Markdown markers (#, ##, etc.) during the PDF-to-Markdown conversion.

The interviewstreet/hiring-agent repository extracts structured data from résumé PDFs by first converting them into Markdown-formatted text. During this conversion, the system distinguishes headings from body text using font metadata analysis, ensuring that sections like "Work Experience" and "Education" are properly marked for downstream LLM parsing.

Entry Point: PDFHandler Orchestrates Extraction

The extraction process begins in pdf.py with the PDFHandler class. The method extract_text_from_pdf opens the PDF using pymupdf (PyMuPDF) and delegates the heavy lifting to the to_markdown function defined in pymupdf_rag.py (around lines 52-58). This creates the bridge between raw PDF bytes and structured Markdown, splitting the workflow into document opening, text span analysis, and final Markdown assembly.

Header Identification Logic

Scanning Document Structure with IdentifyHeaders

Inside pymupdf_rag.py, the IdentifyHeaders helper class scans the entire document to establish font-size baselines before any text is extracted. It provides the get_header_id method, which assigns numeric header IDs to individual text spans based on their font properties relative to the document’s typography. This allows the system to distinguish between a section header and bolded body text.

Determining Heading Levels via max_header_id

The max_header_id(spans, page) function (implemented around lines 24-31) evaluates each line’s spans to determine if it constitutes a heading. The function builds a list of header IDs by calling get_header_id on relevant spans, then selects the smallest non-zero length to determine the appropriate Markdown level. It returns a string of # characters (e.g., #, ##, ###) representing the Markdown heading prefix, or an empty string if no heading is detected.

Marking Headings and Preserving Rich Text

Applying Markdown Prefixes

When max_header_id returns a truthy hdr_string, the line is treated as a heading. The extraction logic (around lines 35-94 of pymupdf_rag.py) prepends this prefix to the line content, transforming raw PDF text like "Work Experience" into Markdown-formatted ## Work Experience.

Handling Inline Formatting and Links

The system preserves rich text during conversion. When headings contain links, each span is processed to maintain bold, italic, monospaced, or strike-through styles before the Markdown link syntax is applied. For plain headings (no links), the text is wrapped with the heading prefix and standard formatting is applied inline. This ensures that visual hierarchy and hyperlinks survive the transition from PDF to Markdown.

Practical Implementation Example

from pdf import PDFHandler

handler = PDFHandler()
pdf_path = "samples/resume.pdf"

# Extract the whole résumé as a structured JSON object.

resume_json = handler.extract_json_from_pdf(pdf_path)
print(resume_json.basics.name)          # → Jane Doe

print(resume_json.work[0].position)    # → Senior Software Engineer

Under the hood, the call chain flows through:

PDFHandler.extract_json_from_pdf
 └─ PDFHandler.extract_text_from_pdf
      └─ pymupdf.open → to_markdown
          └─ max_header_id (detects headings)
          └─ resolve_links (adds Markdown links)
          └─ builds Markdown text with heading prefixes

Summary

• The PDFHandler class in pdf.py initiates the extraction pipeline by calling to_markdown from pymupdf_rag.py.
• IdentifyHeaders and max_header_id analyze font sizes to detect heading hierarchies and determine the appropriate Markdown level.
• Valid headings are prefixed with # markers corresponding to their detected level (1 through 6).
• Inline formatting (bold, italic) and hyperlinks are preserved during the conversion process.
• The resulting Markdown is passed to LLM parsers defined in transform.py for final JSON structuring into data classes like JSONResume.

Frequently Asked Questions

Which file contains the heading detection logic?

The heading detection logic resides in pymupdf_rag.py, specifically within the max_header_id function and the IdentifyHeaders helper class. These components analyze font metadata to distinguish headings from regular text based on document-wide heuristics.

How does the Hiring Agent determine whether a line is an H1 or H2?

The system uses font-size comparison. The max_header_id function evaluates all spans within a line and selects the smallest non-zero header ID returned by get_header_id, converting that numeric level into the corresponding number of # symbols for Markdown (e.g., level 2 becomes ##).

Does the extraction process preserve formatting inside headings?

Yes. When processing headings, the code explicitly handles bold, italic, monospaced, and strike-through styles. If links are present, they are converted to Markdown link syntax while preserving the inline character formatting from the original PDF.

What is the entry point for extracting text from a PDF?

The PDFHandler.extract_text_from_pdf method in pdf.py serves as the primary entry point. It opens the PDF with pymupdf and invokes the to_markdown function to perform the conversion, returning raw Markdown text that is subsequently parsed into structured JSON.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}