# What Prompts Does the Hiring-Agent Use for Developer Interview Evaluation?

> Discover the Jinja-based prompts InterviewStreet's hiring-agent uses to evaluate developer candidates. Learn how JSON output schemas ensure deterministic parsing.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: internals
- Published: 2026-07-04

---

**The hiring-agent uses Jinja-based templates stored in `prompts/templates/` to extract resume sections and evaluate candidates, with each prompt enforcing strict JSON output schemas for deterministic parsing.**

The **interviewstreet/hiring-agent** repository automates developer candidate evaluation through a structured prompt engineering system. Understanding how these **hiring-agent prompts for developer interviews** are organized helps clarify how the system extracts structured data from unstructured PDF resumes and generates consistent scoring evaluations.

## Prompt Architecture and Organization

### Template Structure and Location

All prompt files reside under `prompts/templates/` and follow a declarative Jinja2 syntax. The repository separates concerns by creating individual templates for each resume section and evaluation phase:

- **`basics.jinja`** – Extracts candidate name, contact information, location, and social profile URLs from markdown-formatted resumes
- **`work.jinja`** – Parses employment history including job titles, companies, dates, descriptions, and key highlights
- **`education.jinja`** – Captures academic degrees, institutions, dates, and relevant coursework
- **`skills.jinja`** – Identifies programming languages, frameworks, tools, and expertise classifications
- **`projects.jinja`** – Extracts personal or open-source project details including tech stacks and URLs
- **`awards.jinja`** – Records honors, competitions, and certifications

### TemplateManager Implementation

The **`TemplateManager`** class in [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py) initializes a Jinja `Environment` and exposes the `render_template(section_name, ...)` method. This abstraction ensures that prompt text remains decoupled from Python logic, allowing modifications to wording or JSON field requirements without touching the extraction code.

## Types of Prompts Used in Developer Interview Evaluation

### Resume Section Extraction Prompts

Each section template accepts a `text_content` variable containing the raw markdown resume extracted via `pymupdf_rag.to_markdown`. The prompts instruct the language model to return **only valid JSON** matching the Pydantic schemas defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). For example, the `work.jinja` template directs the LLM to structure employment data as a list of objects with specific required fields.

### System Message Templates

Two critical system templates establish model behavior:

- **`system_message.jinja`** – Provides the "you are a helpful assistant" preamble used before each section extraction request
- **`resume_evaluation_system_message.jinja`** – Contains the core scoring instructions that enforce fairness rules, score caps, and strict JSON output requirements during final candidate evaluation

### Resume Evaluation and Scoring Prompts

The evaluation phase combines `resume_evaluation_system_message.jinja` with **`resume_evaluation_criteria.jinja`**. This pairing defines the scoring dimensions specific to developer interviews, including:

- **Open-source contributions**
- **Self-initiated projects**
- **Production experience**
- **Technical skills assessment**

These templates explicitly define numeric limits and validation constraints that the evaluation model must obey, ensuring consistent scoring across all candidates.

### GitHub Project Selection Prompts

The **`github_project_selection.jinja`** template supports the [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) module by instructing the LLM to identify and rank the top 7 most relevant repositories from a candidate's GitHub profile. This automated curation helps focus technical interviews on the most significant code samples.

## How Prompts Are Executed in the Pipeline

The hiring-agent processes developer interviews through a three-stage pipeline:

1. **PDF Conversion** – `PDFHandler` uses `pymupdf_rag.to_markdown` to convert PDF resumes into plain markdown text
2. **Section Extraction** – The handler calls `TemplateManager.render_template()` for each resume section, constructing system + user message pairs sent to the LLM via `self.provider.chat`
3. **Scoring** – After assembling a `JSONResume` object, [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) renders the evaluation templates and requests a JSON score object that respects the strict constraints defined in the criteria template

All outputs undergo validation through `json.loads` and Pydantic models to ensure deterministic parsing regardless of whether the backend uses Ollama or Google Gemini.

## Code Examples

### Rendering a Prompt Template

```python
from prompts.template_manager import TemplateManager

tm = TemplateManager()                # loads all .jinja files from prompts/templates/

resume_md = "... markdown extracted from PDF ..."
work_prompt = tm.render_template("work", text_content=resume_md)
print(work_prompt)                    # → prompt string sent to the LLM

```

*Source:* [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py)

### Extracting Resume Sections via PDFHandler

```python
from pdf import PDFHandler

handler = PDFHandler()
pdf_path = "candidate_resume.pdf"
json_resume = handler.extract_json_from_pdf(pdf_path)   # full pipeline using work.jinja, skills.jinja, etc.

print(json_resume.work)                                # list of Work objects validated by Pydantic

```

*Source:* [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py)

### Running the Final Evaluation

```python
from evaluator import evaluate_resume
from models import JSONResume

# Assume json_resume was produced by PDFHandler above

score_payload = evaluate_resume(json_resume)   # uses resume_evaluation_system_message.jinja

print(score_payload.candidate_name, score_payload.scores)

```

*Source:* [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)

## Summary

- The **hiring-agent** uses **Jinja2 templates** stored in `prompts/templates/` to maintain separation between prompt engineering and application logic
- **Section-specific templates** (`basics.jinja`, `work.jinja`, `skills.jinja`, etc.) handle structured extraction from markdown resumes
- **Evaluation templates** enforce strict JSON schemas with score caps and fairness rules for consistent developer assessment
- The **`TemplateManager`** class provides a provider-agnostic interface that works with both Ollama and Google Gemini backends
- All prompts include explicit "Return ONLY valid JSON" instructions to ensure deterministic parsing through Pydantic models

## Frequently Asked Questions

### How does the hiring-agent ensure consistent JSON output from the LLM?

Each template in `prompts/templates/` includes explicit instructions stating "**IMPORTANT: Return ONLY valid JSON**" and references the specific schema definitions in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). The system then validates all LLM responses using `json.loads` and Pydantic models, raising errors if the output deviates from the expected structure.

### Can I modify the evaluation criteria without changing Python code?

Yes. The `resume_evaluation_criteria.jinja` file contains the scoring dimensions and numeric limits as template variables. Editing this file changes how the LLM evaluates candidates for open-source contributions, production experience, and technical skills without requiring modifications to [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py).

### What is the difference between `system_message.jinja` and `resume_evaluation_system_message.jinja`?

The `system_message.jinja` provides a general "helpful assistant" preamble used during resume section extraction, while `resume_evaluation_system_message.jinja` contains specialized instructions for the scoring phase, including fairness rules, score caps, and specific validation requirements for the final evaluation JSON.

### How does the system handle different LLM providers?

The `TemplateManager` class generates raw prompt strings that are passed to the provider interface defined in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). Because the Jinja templates produce provider-agnostic text, the same `work.jinja` or `resume_evaluation_criteria.jinja` templates work interchangeably with Ollama, Google Gemini, or other supported backends without modification.