How the InterviewStreet Hiring-Agent Handles Candidate Assessments: A Technical Deep Dive

The hiring-agent automates candidate assessments by converting PDF résumés into structured JSONResume objects, enriching them with GitHub data, and scoring them via LLM-driven evaluation across categories like open-source contributions and technical skills.

The interviewstreet/hiring-agent repository provides an end-to-end pipeline for technical recruiting teams. It processes unstructured résumé documents through a multi-stage architecture that combines document parsing, external data enrichment, and configurable large language model (LLM) inference to produce quantified candidate assessments.

From PDF to Structured Data: The Résumé Parsing Pipeline

The assessment workflow begins with document ingestion and structured extraction.

Extracting Text with PDFHandler

The PDFHandler class in pdf.py handles the initial document processing. It uses PyMuPDF to read candidate PDFs and convert each page to markdown format via the to_markdown method.

The extraction process operates in two phases:

• extract_text_from_pdf (lines 47–62): Reads the raw PDF content and converts it to markdown text.
• _extract_all_sections_separately (lines 66–104): Prompts the LLM with section-specific templates to extract discrete résumé components including basics, work experience, education, skills, projects, and awards.

Normalizing to JSONResume

Raw LLM output requires normalization before evaluation. The transform_parsed_data function in transform.py (lines 6–30) converts the extracted JSON into the canonical JSONResume model defined in models.py.

This transformation merges fragmented work experiences, standardizes date formats, and extracts usernames from profile URLs to ensure consistent data structures for downstream processing.

Enriching Profiles with External Data

Beyond the résumé itself, the pipeline augments candidate profiles with live portfolio data.

GitHub Portfolio Integration

When the JSONResume.basics.profiles field contains a GitHub URL, the system triggers fetch_and_display_github_info in github.py (referenced from score.py line 73). This function retrieves public profile information and recent projects, then convert_github_data_to_text appends a formatted GitHub section to the résumé text.

This enrichment provides the LLM with evidence of open-source contributions and coding activity that candidates may not explicitly detail in their PDF documents.

LLM-Based Evaluation Engine

The core assessment logic resides in the ResumeEvaluator class in evaluator.py, which orchestrates the LLM interaction.

Prompt Construction and Templates

The evaluator constructs prompts using Jinja2 templates stored in prompts/templates/:

• resume_evaluation_system_message.jinja: Defines the LLM's role and output constraints.
• resume_evaluation_criteria.jinja: Embeds the résumé text and specific scoring rubrics.

The _load_evaluation_prompt method (lines 40–46) loads these templates and injects the candidate's enriched résumé text, creating a comprehensive evaluation context.

Model Provider Selection

Provider configuration is managed in prompt.py (lines 15–44), which defines:

• DEFAULT_MODEL: The default LLM identifier.
• MODEL_PROVIDER_MAPPING: Maps model names to provider classes (OllamaProvider or GeminiProvider).
• MODEL_PARAMETERS: Temperature and top-p settings per model.

The initialize_llm_provider function in llm_utils.py instantiates the appropriate provider based on these mappings.

Structured Evaluation Output

The evaluate_resume method (lines 78–86) calls the provider with a structured output constraint:

provider.chat(
    messages=messages,
    format=EvaluationData.model_json_schema()
)

This ensures the LLM returns JSON that validates against the EvaluationData Pydantic model, containing scores for open_source, self_projects, production, and technical_skills, plus bonus points and deductions.

Score Calculation and Reporting

The score.py module orchestrates the final scoring logic and output generation.

Aggregation Logic

The _evaluate_resume function (lines 62–85) processes the EvaluationData object:

1. Sums category scores (each capped at individual maximums).
2. Adds bonus points (capped at 20).
3. Subtracts deductions.
4. Enforces a total score ceiling of 120 and floor of -20 (via constants in evaluator.py).

Output Generation

The print_evaluation_results function (lines 28–74) generates human-readable reports displaying the overall score, category breakdowns, key strengths, and improvement areas. Additionally, the system appends a CSV row to resume_evaluations.csv for batch analysis and downstream hiring analytics.

Running the Assessment Pipeline

You can execute candidate assessments via CLI or integrate the library programmatically.

Command-Line Interface

Run a single assessment from the repository root:

python score.py path/to/candidate_resume.pdf

This prints the formatted evaluation and updates the CSV log.

Programmatic Integration

Import the scoring logic directly into Python applications:

from score import main as assess_resume

# Returns an EvaluationData instance or None on failure

evaluation = assess_resume("candidate_resume.pdf")
print(evaluation.scores.open_source.score)   # e.g., 28.5

print(evaluation.key_strengths)             # e.g., ['Leadership', 'Problem solving']

For direct evaluation bypassing PDF handling:

from evaluator import ResumeEvaluator
from transform import convert_json_resume_to_text

# Assuming `resume` is a JSONResume object

resume_text = convert_json_resume_to_text(resume)
evaluator = ResumeEvaluator(model_name="gemma3:4b")
evaluation_data = evaluator.evaluate_resume(resume_text)

Summary

• The PDFHandler in pdf.py extracts and sections résumé text using PyMuPDF and LLM-based parsing templates.
• Transform logic in transform.py normalizes raw output into the canonical JSONResume model.
• GitHub enrichment via github.py augments profiles with live coding portfolio data.
• The ResumeEvaluator in evaluator.py constructs detailed prompts and calls configurable LLM providers (Ollama or Gemini).
• Scoring logic in score.py aggregates category scores, applies bonuses/deductions, and enforces score boundaries (max 120, min -20).
• The system supports both CLI execution and programmatic integration for flexible deployment in hiring workflows.

Frequently Asked Questions

How does the hiring-agent extract data from PDF résumés?

The system uses the PDFHandler class in pdf.py, specifically extract_text_from_pdf (lines 47–62) to convert PDFs to markdown, then _extract_all_sections_separately (lines 66–104) to parse sections via LLM prompts with specific templates for basics, work, education, and skills.

What external data sources does the hiring-agent use for candidate assessments?

The pipeline enriches résumés with GitHub data when profile URLs are detected. The fetch_and_display_github_info function in github.py retrieves public repositories and profile metadata, appending them to the evaluation context via convert_github_data_to_text.

How is the final candidate score calculated?

The _evaluate_resume function in score.py (lines 62–85) sums category scores from EvaluationData, adds bonus points (capped at 20), subtracts deductions, and enforces boundaries between -20 and 120 as defined in evaluator.py constants.

Can I use a different LLM provider for evaluations?

Yes. Configure the desired model in prompt.py using DEFAULT_MODEL and MODEL_PROVIDER_MAPPING (lines 15–44). The system supports Ollama (local) and Gemini (cloud) providers through the initialize_llm_provider factory in llm_utils.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}