How the Hiring-Agent Processes Candidate Responses Based on Prompts

The hiring-agent evaluates candidates by parsing raw résumé data into structured models, rendering Jinja-based prompt templates, and invoking a configurable LLM provider to return structured evaluation scores.

The interviewstreet/hiring-agent repository implements a robust pipeline to evaluate candidate submissions using large language models. Understanding how it processes candidate responses based on prompts reveals a sophisticated workflow that bridges data transformation, template rendering, and provider-agnostic LLM inference through a sequence of discrete, traceable steps.

Parse and Transform Raw Input

The pipeline begins by normalizing unstructured candidate data into a strict type system before any prompt construction occurs.

Input Parsing

An external parser (e.g., jsonresume-parser) first produces a loosely-structured Python dictionary representing the candidate's submission.

Data Normalization with transform_parsed_data

The transform_parsed_data function in transform.py converts the loose dictionary into the strict JSONResume model defined in models.py. This normalization handles missing sections, converts date ranges via parse_date_range, maps social profile URLs to platform names using extract_domain_from_url and get_network_name, and builds uniform lists for work history, education, skills, and projects.

Result: a fully-typed JSONResume instance ready for textual conversion.

Convert Structured Resume to Plain Text

Once normalized, the structured data must become LLM-readable text. The convert_json_resume_to_text function (also in transform.py) walks the JSONResume object and emits a human-readable string organized into sections like BASIC INFORMATION, WORK EXPERIENCE, and PROJECTS. This text serves as the user-side content in the final prompt.

Load and Render Prompt Templates

The evaluation relies on two distinct Jinja templates managed by TemplateManager in prompts/template_manager.py, which lazily loads templates and provides render_template(section_name, **kwargs).

Template	Purpose	Location
resume_evaluation_criteria.jinja	Contains scoring rules, fairness constraints, and the JSON skeleton the model must fill.	prompts/templates/resume_evaluation_criteria.jinja
resume_evaluation_system_message.jinja	Sets the model's role (e.g., "You are a senior hiring engineer...").	prompts/templates/resume_evaluation_system_message.jinja

During evaluation, the system renders both templates:

criteria_prompt = self.template_manager.render_template(
    "resume_evaluation_criteria", text_content=resume_text
)
system_msg = self.template_manager.render_template(
    "resume_evaluation_system_message"
)

Initialize the LLM Provider

The initialize_llm_provider function in llm_utils.py selects the appropriate backend based on the model name:

• Ollama (default) → OllamaProvider defined in models.py
• Google Gemini → GeminiProvider (also in models.py) when detected in MODEL_PROVIDER_MAPPING

The LLM receives configuration options like temperature and top_p from prompt.py.

Execute Evaluation and Parse Response

The ResumeEvaluator.evaluate_resume method in evaluator.py orchestrates the final LLM call:

chat_params = {
    "model": self.model_name,
    "messages": [
        {"role": "system", "content": system_msg},
        {"role": "user", "content": criteria_prompt},
    ],
    "options": {
        "stream": False,
        "temperature": self.model_params.get("temperature", 0.5),
        "top_p": self.model_params.get("top_p", 0.9),
    },
}
response = self.provider.chat(**chat_params, format=EvaluationData.model_json_schema())

The provider returns a response that may be wrapped in Markdown code fences. The extract_json_from_response function in llm_utils.py strips these fences and any extraneous formatting before the final structured evaluation is returned.

Summary

• Data transformation in transform.py normalizes raw input into strict JSONResume models via transform_parsed_data.
• Text conversion via convert_json_resume_to_text prepares LLM-readable content.
• Template rendering through TemplateManager in prompts/template_manager.py injects candidate data into Jinja templates.
• Provider initialization in llm_utils.py supports both Ollama and Gemini backends.
• Evaluation execution in evaluator.py sends structured chat parameters and enforces JSON schema output.

Frequently Asked Questions

How does the hiring-agent handle different résumé formats?

The system relies on an external parser to initially convert varied formats into a loose Python dictionary. The transform_parsed_data function in transform.py then normalizes this into the strict JSONResume model, handling missing fields, standardizing date ranges, and mapping social URLs regardless of the original input format.

What prompt templates does the hiring-agent use?

The system uses two Jinja templates stored in prompts/templates/: resume_evaluation_criteria.jinja contains the scoring rules and JSON output schema, while resume_evaluation_system_message.jinja establishes the model's persona and constraints. The TemplateManager class renders these with candidate-specific data at runtime.

Can the hiring-agent work with different LLM providers?

Yes. The initialize_llm_provider function in llm_utils.py dynamically selects between OllamaProvider and GeminiProvider based on the model name configuration, making the evaluation pipeline provider-agnostic while maintaining consistent prompt formatting and response handling.

How does the system ensure structured output from the LLM?

The evaluate_resume method in evaluator.py passes EvaluationData.model_json_schema() as the format parameter to the provider's chat method, enforcing JSON schema compliance. Additionally, extract_json_from_response in llm_utils.py sanitizes the raw response by stripping Markdown code fences before parsing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}