What Prompts Does the Hiring-Agent Use for Developer Interview Evaluation?

The hiring-agent uses Jinja-based templates stored in prompts/templates/ to extract resume sections and evaluate candidates, with each prompt enforcing strict JSON output schemas for deterministic parsing.

The interviewstreet/hiring-agent repository automates developer candidate evaluation through a structured prompt engineering system. Understanding how these hiring-agent prompts for developer interviews are organized helps clarify how the system extracts structured data from unstructured PDF resumes and generates consistent scoring evaluations.

Prompt Architecture and Organization

Template Structure and Location

All prompt files reside under prompts/templates/ and follow a declarative Jinja2 syntax. The repository separates concerns by creating individual templates for each resume section and evaluation phase:

• basics.jinja – Extracts candidate name, contact information, location, and social profile URLs from markdown-formatted resumes
• work.jinja – Parses employment history including job titles, companies, dates, descriptions, and key highlights
• education.jinja – Captures academic degrees, institutions, dates, and relevant coursework
• skills.jinja – Identifies programming languages, frameworks, tools, and expertise classifications
• projects.jinja – Extracts personal or open-source project details including tech stacks and URLs
• awards.jinja – Records honors, competitions, and certifications

TemplateManager Implementation

The TemplateManager class in prompts/template_manager.py initializes a Jinja Environment and exposes the render_template(section_name, ...) method. This abstraction ensures that prompt text remains decoupled from Python logic, allowing modifications to wording or JSON field requirements without touching the extraction code.

Types of Prompts Used in Developer Interview Evaluation

Resume Section Extraction Prompts

Each section template accepts a text_content variable containing the raw markdown resume extracted via pymupdf_rag.to_markdown. The prompts instruct the language model to return only valid JSON matching the Pydantic schemas defined in models.py. For example, the work.jinja template directs the LLM to structure employment data as a list of objects with specific required fields.

System Message Templates

Two critical system templates establish model behavior:

• system_message.jinja – Provides the "you are a helpful assistant" preamble used before each section extraction request
• resume_evaluation_system_message.jinja – Contains the core scoring instructions that enforce fairness rules, score caps, and strict JSON output requirements during final candidate evaluation

Resume Evaluation and Scoring Prompts

The evaluation phase combines resume_evaluation_system_message.jinja with resume_evaluation_criteria.jinja. This pairing defines the scoring dimensions specific to developer interviews, including:

• Open-source contributions
• Self-initiated projects
• Production experience
• Technical skills assessment

These templates explicitly define numeric limits and validation constraints that the evaluation model must obey, ensuring consistent scoring across all candidates.

GitHub Project Selection Prompts

The github_project_selection.jinja template supports the github.py module by instructing the LLM to identify and rank the top 7 most relevant repositories from a candidate's GitHub profile. This automated curation helps focus technical interviews on the most significant code samples.

How Prompts Are Executed in the Pipeline

The hiring-agent processes developer interviews through a three-stage pipeline:

1. PDF Conversion – PDFHandler uses pymupdf_rag.to_markdown to convert PDF resumes into plain markdown text
2. Section Extraction – The handler calls TemplateManager.render_template() for each resume section, constructing system + user message pairs sent to the LLM via self.provider.chat
3. Scoring – After assembling a JSONResume object, evaluator.py renders the evaluation templates and requests a JSON score object that respects the strict constraints defined in the criteria template

All outputs undergo validation through json.loads and Pydantic models to ensure deterministic parsing regardless of whether the backend uses Ollama or Google Gemini.

Code Examples

Rendering a Prompt Template

from prompts.template_manager import TemplateManager

tm = TemplateManager()                # loads all .jinja files from prompts/templates/

resume_md = "... markdown extracted from PDF ..."
work_prompt = tm.render_template("work", text_content=resume_md)
print(work_prompt)                    # → prompt string sent to the LLM

Source: prompts/template_manager.py

Extracting Resume Sections via PDFHandler

from pdf import PDFHandler

handler = PDFHandler()
pdf_path = "candidate_resume.pdf"
json_resume = handler.extract_json_from_pdf(pdf_path)   # full pipeline using work.jinja, skills.jinja, etc.

print(json_resume.work)                                # list of Work objects validated by Pydantic

Source: pdf.py

Running the Final Evaluation

from evaluator import evaluate_resume
from models import JSONResume

# Assume json_resume was produced by PDFHandler above

score_payload = evaluate_resume(json_resume)   # uses resume_evaluation_system_message.jinja

print(score_payload.candidate_name, score_payload.scores)

Source: evaluator.py

Summary

• The hiring-agent uses Jinja2 templates stored in prompts/templates/ to maintain separation between prompt engineering and application logic
• Section-specific templates (basics.jinja, work.jinja, skills.jinja, etc.) handle structured extraction from markdown resumes
• Evaluation templates enforce strict JSON schemas with score caps and fairness rules for consistent developer assessment
• The TemplateManager class provides a provider-agnostic interface that works with both Ollama and Google Gemini backends
• All prompts include explicit "Return ONLY valid JSON" instructions to ensure deterministic parsing through Pydantic models

Frequently Asked Questions

How does the hiring-agent ensure consistent JSON output from the LLM?

Each template in prompts/templates/ includes explicit instructions stating "IMPORTANT: Return ONLY valid JSON" and references the specific schema definitions in models.py. The system then validates all LLM responses using json.loads and Pydantic models, raising errors if the output deviates from the expected structure.

Can I modify the evaluation criteria without changing Python code?

Yes. The resume_evaluation_criteria.jinja file contains the scoring dimensions and numeric limits as template variables. Editing this file changes how the LLM evaluates candidates for open-source contributions, production experience, and technical skills without requiring modifications to evaluator.py.

What is the difference between system_message.jinja and resume_evaluation_system_message.jinja?

The system_message.jinja provides a general "helpful assistant" preamble used during resume section extraction, while resume_evaluation_system_message.jinja contains specialized instructions for the scoring phase, including fairness rules, score caps, and specific validation requirements for the final evaluation JSON.

How does the system handle different LLM providers?

The TemplateManager class generates raw prompt strings that are passed to the provider interface defined in prompt.py. Because the Jinja templates produce provider-agnostic text, the same work.jinja or resume_evaluation_criteria.jinja templates work interchangeably with Ollama, Google Gemini, or other supported backends without modification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}