How to Customize Resume Evaluation Criteria Templates in the Hiring Agent

You customize resume evaluation criteria by editing the Jinja template at prompts/templates/resume_evaluation_criteria.jinja and restarting the evaluator, with optional dynamic variables passed via TemplateManager.render_template().

The interviewstreet/hiring-agent repository provides an open-source framework for automated resume screening using Large Language Models. Customizing the resume evaluation criteria templates allows you to modify scoring rubrics, adjust point allocations, or add new assessment dimensions to align with your organization's specific hiring requirements.

How the Template System Works

The hiring-agent uses a two-stage templating architecture to construct LLM prompts.

Template Loading Phase: The TemplateManager class in prompts/template_manager.py initializes a Jinja2 Environment with a FileSystemLoader pointing to the prompts/templates directory. During instantiation, it calls _load_templates() to register available templates.

# From prompts/template_manager.py

self.env = Environment(loader=FileSystemLoader(template_dir), ...)
self._load_templates()

Prompt Rendering Phase: When ResumeEvaluator.evaluate_resume() executes in evaluator.py, it requests the resume_evaluation_criteria template from the manager and injects the raw resume text.

# From evaluator.py

criteria_template = self.template_manager.render_template(
    "resume_evaluation_criteria", text_content=resume_text
)

Because templates are plain Jinja files, you modify evaluation logic by editing the source template rather than changing Python code.

Step-by-Step Template Customization

Follow these steps to modify the default scoring criteria.

1. Locate the Template File

Open prompts/templates/resume_evaluation_criteria.jinja in your editor. This file contains the static prompt text that defines scoring categories, point ranges, and evaluation instructions sent to the LLM.

2. Modify Scoring Criteria

Edit the static content to adjust point values or add new sections. For example, to increase the maximum points for Open Source contributions from 35 to 40:

### Before

### Open Source (0-35 points)

### After

### Open Source (0-40 points)

You can add entirely new evaluation sections using standard markdown syntax within the Jinja file.

3. Restart the Service

The TemplateManager loads templates during initialization. After saving changes to the Jinja file, restart the Python process or re-instantiate ResumeEvaluator to pick up the modifications.

Injecting Dynamic Variables

For organization-specific weights or configurable thresholds, pass dynamic data into the template.

Add Placeholders to the Template

Insert Jinja variables using double curly braces:

### Open Source (0-{{ open_source_max }} points)

Pass Variables at Runtime

Update the render_template() call in evaluator.py to include the variable:

criteria_template = self.template_manager.render_template(
    "resume_evaluation_criteria",
    text_content=resume_text,
    open_source_max=40
)

Creating Custom Evaluation Templates

To create entirely new evaluation frameworks beyond the default criteria:

1. Create the template file: Add prompts/templates/custom_criteria.jinja with your custom evaluation logic.

2. Register the template: Update the dictionary in TemplateManager._load_templates() to map a template name to your file:

# In prompts/template_manager.py

"my_custom_criteria": "my_custom_criteria.jinja",

3. Use in evaluation: Reference the new template name in your evaluator logic:

custom_prompt = self.template_manager.render_template(
    "my_custom_criteria", text_content=resume_text
)

Verification and Testing

Validate your changes by running a quick evaluation in a Python REPL:

from evaluator import ResumeEvaluator

evaluator = ResumeEvaluator()
sample_resume = "John Doe\n\nSkills: Python, Rust\nProjects: Open source contributor..."

result = evaluator.evaluate_resume(sample_resume)
print(result.json(indent=2))

Inspect the JSON output to confirm your modified criteria and scoring ranges appear correctly in the LLM response.

Summary

• Edit the Jinja template at prompts/templates/resume_evaluation_criteria.jinja to modify static scoring rules.
• Restart the service after template changes to reload the FileSystemLoader.
• Pass dynamic variables via TemplateManager.render_template() for configurable weights.
• Register new templates in the _load_templates() dictionary before use.
• Verify changes using the ResumeEvaluator class with sample resume data.

Frequently Asked Questions

Do I need to restart the application after editing a template?

Yes. The TemplateManager loads templates from disk when instantiated. In a long-running service, you must restart the Python process or create a new TemplateManager instance to pick up file changes.

Can I add new scoring categories to the evaluation?

Yes. You can add new sections, headings, or criteria directly in the resume_evaluation_criteria.jinja file. The LLM will evaluate candidates against any rubric defined in the template, provided you follow consistent formatting.

How do I pass company-specific weights to the template?

Add Jinja placeholders like {{ company_weight }} in your template file, then pass the value when calling render_template() in evaluator.py. You can also subclass TemplateManager to inject default context automatically.

Is the template system safe from code injection?

Yes. The system only renders Jinja templates with static prompts and injected resume text. No user-provided code executes during template rendering, preventing injection attacks while allowing flexible prompt customization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}