How Fairness Constraints Are Implemented in the Evaluation Templates of the Hiring-Agent Repository

Fairness constraints are enforced through explicit "CRITICAL FAIRNESS REQUIREMENTS" blocks embedded in both the system and user message Jinja templates, which strictly prohibit demographic and academic bias while mandating evaluation based solely on technical evidence.

The interviewstreet/hiring-agent repository automates resume evaluation using a large language model (LLM) that is governed by strictly defined fairness protocols. This article examines how fairness constraints are implemented in the evaluation templates to guarantee that candidate scoring depends exclusively on technical merit—such as code quality, project impact, and problem-solving approach—rather than personal identifiers or institutional pedigree.

The Dual-Template Architecture for Fairness Enforcement

The repository implements a defense-in-depth strategy using two complementary Jinja templates that embed fairness rules directly into the LLM's prompt context. Because these constraints exist at the prompt level, they cannot be bypassed by the model during inference.

System Message Template (resume_evaluation_system_message.jinja)

Located at prompts/templates/resume_evaluation_system_message.jinja, this template defines the LLM's system role and houses the primary fairness directive. It contains a "CRITICAL FAIRNESS REQUIREMENTS" block that explicitly lists prohibited scoring factors. According to the source code, the system message mandates that scores must never depend on:

• Candidate's name, gender, or any personal demographic information
• College, university, or educational institution name
• CGPA, GPA, or academic grades
• City, location, or geographical information
• Any personal characteristics unrelated to technical skills and experience

The template simultaneously reinforces the evaluation focus on legitimate technical signals: technical skills, project complexity, open-source contributions, work experience, technical communication, and problem-solving capabilities.

User Message Template (resume_evaluation_criteria.jinja)

The second layer of protection resides in prompts/templates/resume_evaluation_criteria.jinja. This template expands the prompt with concrete scoring rubrics while repeating the same "CRITICAL FAIRNESS REQUIREMENTS" section to reinforce compliance. It enumerates category-specific scoring criteria for open-source work, self projects, production experience, and technical skills, explicitly tying scores to fair signals such as:

• Project impact and measurable outcomes
• Open-source contribution type (code vs. documentation vs. issue triage)
• Verified links to repositories or technical blog posts

When the evaluator builds the final prompt, it renders this template with the raw resume text, ensuring the fairness rules are included verbatim in the user-facing portion of the interaction.

Implementation Flow in evaluator.py

The Python orchestration layer in evaluator.py binds these templates together into a complete prompt chain that is sent to the LLM.

The evaluate_resume Method

The ResumeEvaluator.evaluate_resume method initiates the evaluation workflow. It first loads the system message by calling template_manager.render_template("resume_evaluation_system_message"), establishing the fairness constraints as hard-coded instructions for the LLM session.

The _load_evaluation_prompt Helper

Inside ResumeEvaluator._load_evaluation_prompt, the method renders the criteria template with the candidate's resume text:

criteria_template = self.template_manager.render_template(
    "resume_evaluation_criteria", text_content=resume_text
)

# The rendered template includes the fairness block (see file)

return criteria_template

Both rendered strings—the system message and the user message—are sent to the LLM as a paired message sequence. The LLM, constrained by the fairness directives present in both messages, produces a JSON evaluation that respects the defined limits, including caps on bonuses and category-specific score ceilings.

Validating Fairness with Structured Output

The models.py file defines the EvaluationData Pydantic model, which validates the LLM's JSON output. This schema enforces structure on the response, ensuring that the evaluation contains only the permitted technical categories and score ranges defined in the templates. By combining prompt-level constraints with schema validation, the system creates a closed loop where fairness violations are structurally difficult to produce.

Practical Example: Running a Fair Evaluation

To evaluate a resume using these fairness constraints, instantiate the evaluator and process the text:

from evaluator import ResumeEvaluator

# Initialise with the default model (e.g., Gemini)

evaluator = ResumeEvaluator()

# `resume_text` is the raw resume string supplied by the user

evaluation = evaluator.evaluate_resume(resume_text)

print(evaluation.json())

This invocation automatically includes the fairness requirements from both Jinja templates, ensuring the LLM ignores prohibited signals like candidate name or university reputation while scoring technical accomplishments.

Summary

• Dual-template injection: Fairness constraints are embedded in both resume_evaluation_system_message.jinja (system role) and resume_evaluation_criteria.jinja (user prompt), creating redundant guardrails against bias.
• Explicit prohibition: The templates explicitly ban scoring based on name, gender, college, CGPA, or location, limiting evaluation to technical skills, project complexity, and verified contributions.
• Orchestration in evaluator.py: The ResumeEvaluator.evaluate_resume method coordinates the rendering of both templates, while _load_evaluation_prompt injects the raw resume into the fairness-aware criteria template.
• Validation layer: The EvaluationData model in models.py structures the LLM output to ensure compliance with the permitted evaluation categories.
• Immutable constraints: Because fairness rules are hard-coded in the prompt templates rather than application logic, they cannot be circumvented through API parameters or user input.

Frequently Asked Questions

What specific demographic factors are excluded from resume scoring?

The templates explicitly prohibit scoring based on the candidate's name, gender, personal demographic information, college or university name, CGPA/GPA/academic grades, city, location, geographical information, and any personal characteristics unrelated to technical skills. The LLM is instructed to ignore these signals entirely and focus only on technical evidence present in the resume and supplemental GitHub or blog data.

How does the system prevent the LLM from ignoring fairness rules?

The implementation uses a defense-in-depth approach: the "CRITICAL FAIRNESS REQUIREMENTS" block appears in both the system message (which sets the LLM's role) and the user message (which contains the specific evaluation instructions). This dual placement ensures the constraints are present in both the high-level instructions and the immediate task context, making it statistically improbable for the model to overlook them. Additionally, the structured EvaluationData schema in models.py limits the output format to predefined technical categories.

Can the fairness constraints be modified or disabled?

The fairness rules are embedded as static text within the Jinja templates located in prompts/templates/. To modify the constraints, you would need to edit resume_evaluation_system_message.jinja or resume_evaluation_criteria.jinja directly. There is no runtime flag to disable fairness constraints, as removing them would require altering the template files themselves, ensuring that fairness remains a core architectural property rather than an optional configuration.

Where does the template rendering logic reside?

The template loading and rendering logic is implemented in prompts/template_manager.py, which is invoked by ResumeEvaluator in evaluator.py. The template_manager.render_template method takes the template name and context variables (such as text_content=resume_text) and returns the rendered string that includes the embedded fairness requirements.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}