# How the InterviewStreet Hiring-Agent Handles Candidate Assessments: A Technical Deep Dive

> Explore the interviewstreet hiring agent's technical process for candidate assessments. Learn how it converts résumés, enriches data, and scores skills using LLMs to streamline hiring.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: deep-dive
- Published: 2026-07-12

---

**The hiring-agent automates candidate assessments by converting PDF résumés into structured JSONResume objects, enriching them with GitHub data, and scoring them via LLM-driven evaluation across categories like open-source contributions and technical skills.**

The `interviewstreet/hiring-agent` repository provides an end-to-end pipeline for technical recruiting teams. It processes unstructured résumé documents through a multi-stage architecture that combines document parsing, external data enrichment, and configurable large language model (LLM) inference to produce quantified candidate assessments.

## From PDF to Structured Data: The Résumé Parsing Pipeline

The assessment workflow begins with document ingestion and structured extraction.

### Extracting Text with PDFHandler

The **`PDFHandler`** class in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) handles the initial document processing. It uses **PyMuPDF** to read candidate PDFs and convert each page to markdown format via the `to_markdown` method.

The extraction process operates in two phases:

- **`extract_text_from_pdf`** (lines 47–62): Reads the raw PDF content and converts it to markdown text.
- **`_extract_all_sections_separately`** (lines 66–104): Prompts the LLM with section-specific templates to extract discrete résumé components including basics, work experience, education, skills, projects, and awards.

### Normalizing to JSONResume

Raw LLM output requires normalization before evaluation. The **`transform_parsed_data`** function in [`transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/transform.py) (lines 6–30) converts the extracted JSON into the canonical **`JSONResume`** model defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py).

This transformation merges fragmented work experiences, standardizes date formats, and extracts usernames from profile URLs to ensure consistent data structures for downstream processing.

## Enriching Profiles with External Data

Beyond the résumé itself, the pipeline augments candidate profiles with live portfolio data.

### GitHub Portfolio Integration

When the `JSONResume.basics.profiles` field contains a GitHub URL, the system triggers **`fetch_and_display_github_info`** in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) (referenced from [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) line 73). This function retrieves public profile information and recent projects, then **`convert_github_data_to_text`** appends a formatted GitHub section to the résumé text.

This enrichment provides the LLM with evidence of open-source contributions and coding activity that candidates may not explicitly detail in their PDF documents.

## LLM-Based Evaluation Engine

The core assessment logic resides in the **`ResumeEvaluator`** class in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), which orchestrates the LLM interaction.

### Prompt Construction and Templates

The evaluator constructs prompts using Jinja2 templates stored in `prompts/templates/`:

- **`resume_evaluation_system_message.jinja`**: Defines the LLM's role and output constraints.
- **`resume_evaluation_criteria.jinja`**: Embeds the résumé text and specific scoring rubrics.

The **`_load_evaluation_prompt`** method (lines 40–46) loads these templates and injects the candidate's enriched résumé text, creating a comprehensive evaluation context.

### Model Provider Selection

Provider configuration is managed in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) (lines 15–44), which defines:

- **`DEFAULT_MODEL`**: The default LLM identifier.
- **`MODEL_PROVIDER_MAPPING`**: Maps model names to provider classes (`OllamaProvider` or `GeminiProvider`).
- **`MODEL_PARAMETERS`**: Temperature and top-p settings per model.

The **`initialize_llm_provider`** function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) instantiates the appropriate provider based on these mappings.

### Structured Evaluation Output

The **`evaluate_resume`** method (lines 78–86) calls the provider with a structured output constraint:

```python
provider.chat(
    messages=messages,
    format=EvaluationData.model_json_schema()
)

```

This ensures the LLM returns JSON that validates against the **`EvaluationData`** Pydantic model, containing scores for `open_source`, `self_projects`, `production`, and `technical_skills`, plus bonus points and deductions.

## Score Calculation and Reporting

The **[`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py)** module orchestrates the final scoring logic and output generation.

### Aggregation Logic

The **`_evaluate_resume`** function (lines 62–85) processes the `EvaluationData` object:

1. Sums category scores (each capped at individual maximums).
2. Adds bonus points (capped at 20).
3. Subtracts deductions.
4. Enforces a total score ceiling of 120 and floor of -20 (via constants in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)).

### Output Generation

The **`print_evaluation_results`** function (lines 28–74) generates human-readable reports displaying the overall score, category breakdowns, key strengths, and improvement areas. Additionally, the system appends a CSV row to `resume_evaluations.csv` for batch analysis and downstream hiring analytics.

## Running the Assessment Pipeline

You can execute candidate assessments via CLI or integrate the library programmatically.

### Command-Line Interface

Run a single assessment from the repository root:

```bash
python score.py path/to/candidate_resume.pdf

```

This prints the formatted evaluation and updates the CSV log.

### Programmatic Integration

Import the scoring logic directly into Python applications:

```python
from score import main as assess_resume

# Returns an EvaluationData instance or None on failure

evaluation = assess_resume("candidate_resume.pdf")
print(evaluation.scores.open_source.score)   # e.g., 28.5

print(evaluation.key_strengths)             # e.g., ['Leadership', 'Problem solving']

```

For direct evaluation bypassing PDF handling:

```python
from evaluator import ResumeEvaluator
from transform import convert_json_resume_to_text

# Assuming `resume` is a JSONResume object

resume_text = convert_json_resume_to_text(resume)
evaluator = ResumeEvaluator(model_name="gemma3:4b")
evaluation_data = evaluator.evaluate_resume(resume_text)

```

## Summary

- The **PDFHandler** in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) extracts and sections résumé text using PyMuPDF and LLM-based parsing templates.
- **Transform logic** in [`transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/transform.py) normalizes raw output into the canonical `JSONResume` model.
- **GitHub enrichment** via [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) augments profiles with live coding portfolio data.
- The **ResumeEvaluator** in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) constructs detailed prompts and calls configurable LLM providers (Ollama or Gemini).
- **Scoring logic** in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) aggregates category scores, applies bonuses/deductions, and enforces score boundaries (max 120, min -20).
- The system supports both **CLI execution** and **programmatic integration** for flexible deployment in hiring workflows.

## Frequently Asked Questions

### How does the hiring-agent extract data from PDF résumés?

The system uses the `PDFHandler` class in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py), specifically `extract_text_from_pdf` (lines 47–62) to convert PDFs to markdown, then `_extract_all_sections_separately` (lines 66–104) to parse sections via LLM prompts with specific templates for basics, work, education, and skills.

### What external data sources does the hiring-agent use for candidate assessments?

The pipeline enriches résumés with **GitHub data** when profile URLs are detected. The `fetch_and_display_github_info` function in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) retrieves public repositories and profile metadata, appending them to the evaluation context via `convert_github_data_to_text`.

### How is the final candidate score calculated?

The `_evaluate_resume` function in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) (lines 62–85) sums category scores from `EvaluationData`, adds bonus points (capped at 20), subtracts deductions, and enforces boundaries between -20 and 120 as defined in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) constants.

### Can I use a different LLM provider for evaluations?

Yes. Configure the desired model in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) using `DEFAULT_MODEL` and `MODEL_PROVIDER_MAPPING` (lines 15–44). The system supports **Ollama** (local) and **Gemini** (cloud) providers through the `initialize_llm_provider` factory in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py).