# How Fairness Constraints Are Implemented in the Evaluation Templates of the Hiring-Agent Repository

> Discover how fairness constraints are implemented in the hiring-agent evaluation templates. Learn about critical fairness requirements that prevent bias and ensure unbiased technical evaluations.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: internals
- Published: 2026-07-19

---

**Fairness constraints are enforced through explicit "CRITICAL FAIRNESS REQUIREMENTS" blocks embedded in both the system and user message Jinja templates, which strictly prohibit demographic and academic bias while mandating evaluation based solely on technical evidence.**

The interviewstreet/hiring-agent repository automates resume evaluation using a large language model (LLM) that is governed by strictly defined fairness protocols. This article examines how fairness constraints are implemented in the evaluation templates to guarantee that candidate scoring depends exclusively on technical merit—such as code quality, project impact, and problem-solving approach—rather than personal identifiers or institutional pedigree.

## The Dual-Template Architecture for Fairness Enforcement

The repository implements a defense-in-depth strategy using two complementary Jinja templates that embed fairness rules directly into the LLM's prompt context. Because these constraints exist at the prompt level, they cannot be bypassed by the model during inference.

### System Message Template (`resume_evaluation_system_message.jinja`)

Located at `prompts/templates/resume_evaluation_system_message.jinja`, this template defines the LLM's system role and houses the primary fairness directive. It contains a **"CRITICAL FAIRNESS REQUIREMENTS"** block that explicitly lists prohibited scoring factors. According to the source code, the system message mandates that scores must never depend on:

- Candidate's name, gender, or any personal demographic information
- College, university, or educational institution name
- CGPA, GPA, or academic grades
- City, location, or geographical information
- Any personal characteristics unrelated to technical skills and experience

The template simultaneously reinforces the **evaluation focus** on legitimate technical signals: technical skills, project complexity, open-source contributions, work experience, technical communication, and problem-solving capabilities.

### User Message Template (`resume_evaluation_criteria.jinja`)

The second layer of protection resides in `prompts/templates/resume_evaluation_criteria.jinja`. This template expands the prompt with concrete scoring rubrics while repeating the same "CRITICAL FAIRNESS REQUIREMENTS" section to reinforce compliance. It enumerates category-specific scoring criteria for open-source work, self projects, production experience, and technical skills, explicitly tying scores to fair signals such as:

- **Project impact** and measurable outcomes
- **Open-source contribution type** (code vs. documentation vs. issue triage)
- **Verified links** to repositories or technical blog posts

When the evaluator builds the final prompt, it renders this template with the raw resume text, ensuring the fairness rules are included verbatim in the user-facing portion of the interaction.

## Implementation Flow in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)

The Python orchestration layer in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) binds these templates together into a complete prompt chain that is sent to the LLM.

### The `evaluate_resume` Method

The `ResumeEvaluator.evaluate_resume` method initiates the evaluation workflow. It first loads the system message by calling `template_manager.render_template("resume_evaluation_system_message")`, establishing the fairness constraints as hard-coded instructions for the LLM session.

### The `_load_evaluation_prompt` Helper

Inside `ResumeEvaluator._load_evaluation_prompt`, the method renders the criteria template with the candidate's resume text:

```python
criteria_template = self.template_manager.render_template(
    "resume_evaluation_criteria", text_content=resume_text
)

# The rendered template includes the fairness block (see file)

return criteria_template

```

Both rendered strings—the system message and the user message—are sent to the LLM as a paired message sequence. The LLM, constrained by the fairness directives present in both messages, produces a JSON evaluation that respects the defined limits, including caps on bonuses and category-specific score ceilings.

## Validating Fairness with Structured Output

The [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) file defines the `EvaluationData` Pydantic model, which validates the LLM's JSON output. This schema enforces structure on the response, ensuring that the evaluation contains only the permitted technical categories and score ranges defined in the templates. By combining prompt-level constraints with schema validation, the system creates a closed loop where fairness violations are structurally difficult to produce.

## Practical Example: Running a Fair Evaluation

To evaluate a resume using these fairness constraints, instantiate the evaluator and process the text:

```python
from evaluator import ResumeEvaluator

# Initialise with the default model (e.g., Gemini)

evaluator = ResumeEvaluator()

# `resume_text` is the raw resume string supplied by the user

evaluation = evaluator.evaluate_resume(resume_text)

print(evaluation.json())

```

This invocation automatically includes the fairness requirements from both Jinja templates, ensuring the LLM ignores prohibited signals like candidate name or university reputation while scoring technical accomplishments.

## Summary

- **Dual-template injection**: Fairness constraints are embedded in both `resume_evaluation_system_message.jinja` (system role) and `resume_evaluation_criteria.jinja` (user prompt), creating redundant guardrails against bias.
- **Explicit prohibition**: The templates explicitly ban scoring based on name, gender, college, CGPA, or location, limiting evaluation to technical skills, project complexity, and verified contributions.
- **Orchestration in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)**: The `ResumeEvaluator.evaluate_resume` method coordinates the rendering of both templates, while `_load_evaluation_prompt` injects the raw resume into the fairness-aware criteria template.
- **Validation layer**: The `EvaluationData` model in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) structures the LLM output to ensure compliance with the permitted evaluation categories.
- **Immutable constraints**: Because fairness rules are hard-coded in the prompt templates rather than application logic, they cannot be circumvented through API parameters or user input.

## Frequently Asked Questions

### What specific demographic factors are excluded from resume scoring?

The templates explicitly prohibit scoring based on the candidate's name, gender, personal demographic information, college or university name, CGPA/GPA/academic grades, city, location, geographical information, and any personal characteristics unrelated to technical skills. The LLM is instructed to ignore these signals entirely and focus only on technical evidence present in the resume and supplemental GitHub or blog data.

### How does the system prevent the LLM from ignoring fairness rules?

The implementation uses a defense-in-depth approach: the "CRITICAL FAIRNESS REQUIREMENTS" block appears in both the system message (which sets the LLM's role) and the user message (which contains the specific evaluation instructions). This dual placement ensures the constraints are present in both the high-level instructions and the immediate task context, making it statistically improbable for the model to overlook them. Additionally, the structured `EvaluationData` schema in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) limits the output format to predefined technical categories.

### Can the fairness constraints be modified or disabled?

The fairness rules are embedded as static text within the Jinja templates located in `prompts/templates/`. To modify the constraints, you would need to edit `resume_evaluation_system_message.jinja` or `resume_evaluation_criteria.jinja` directly. There is no runtime flag to disable fairness constraints, as removing them would require altering the template files themselves, ensuring that fairness remains a core architectural property rather than an optional configuration.

### Where does the template rendering logic reside?

The template loading and rendering logic is implemented in [`prompts/template_manager.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/template_manager.py), which is invoked by `ResumeEvaluator` in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py). The `template_manager.render_template` method takes the template name and context variables (such as `text_content=resume_text`) and returns the rendered string that includes the embedded fairness requirements.