Fairness Constraints in the Evaluation Prompts: Technical Implementation in interviewstreet/hiring-agent

The fairness constraints in the evaluation prompts enforce strict demographic, educational, and geographic neutrality while restricting scoring to technical merit only.

The interviewstreet/hiring-agent repository encodes explicit fairness constraints in the evaluation prompts to ensure AI-driven résumé scoring remains unbiased. These constraints are hard-coded in the prompt templates and programmatically enforced when the language model evaluates candidates, eliminating influence from personal identifiers and academic pedigree.

Where Fairness Constraints Are Defined

The core fairness logic lives in prompts/templates/resume_evaluation_criteria.jinja. This Jinja2 template contains hard-coded rules under the section "CRITICAL FAIRNESS REQUIREMENTS" that the LLM must obey when generating scores. The template explicitly lists fields and attributes that must never influence the evaluation outcome, serving as a programmatic guardrail against algorithmic bias.

Three Core Fairness Categories

The constraints are organized into three neutrality categories that strictly limit what the model can consider.

Demographic Neutrality

Scores must never be influenced by the candidate's name, gender, age, ethnicity, or any personal demographic data. According to the source template at resume_evaluation_criteria.jinja#L5-L12, the prompt explicitly instructs the LLM to ignore all personal identifiers that could reveal protected characteristics, ensuring the evaluation focuses solely on professional capabilities.

Educational Neutrality

The model must ignore institutional pedigree and academic grading metrics. As defined at resume_evaluation_criteria.jinja#L7-L10, the constraints prohibit the LLM from considering college or university names, as well as CGPA, GPA, or any academic grading information. This prevents bias toward specific educational backgrounds or performance metrics that may not correlate with actual job performance.

Geographic Neutrality

Location-related fields cannot affect scoring outcomes. The template at resume_evaluation_criteria.jinja#L10-L11 explicitly forbids the model from using city, state, country, or any geographical information when computing candidate scores, eliminating regional bias from the evaluation process.

Technical Merit Focus Areas

In addition to stating what must be ignored, the fairness constraints explicitly define what the evaluation must prioritize. According to resume_evaluation_criteria.jinja#L13-L19, the LLM is restricted to scoring based solely on:

• Technical skills and programming languages
• Project complexity and real-world impact
• Open-source contributions and community involvement
• Production-level work experience
• Technical communication (blogs, documentation)
• Problem-solving and algorithmic thinking

This positive framing ensures the model allocates scoring weight exclusively to demonstrable technical competencies rather than proxy variables for demographic or socioeconomic status.

How the Constraints Are Enforced at Runtime

The fairness constraints are not merely documentation; they are programmatically injected into every evaluation cycle. The prompt.py module loads the Jinja template and renders it with the candidate's résumé text, while evaluator.py orchestrates the LLM call and parses the structured results.

# Load the evaluation criteria template from prompts/templates/

template_path = "prompts/templates/resume_evaluation_criteria.jinja"
template = env.get_template(template_path)

# Render the prompt containing hard-coded fairness constraints

prompt = template.render(text_content=resume_text)

# The LLM receives the prompt with explicit fairness requirements

response = llm.generate(prompt)

When rendered, the prompt includes the explicit instruction block:

CRITICAL FAIRNESS REQUIREMENTS
SCORES MUST NEVER DEPEND ON:
- Candidate's name, gender, or personal demographic information
- College, university, or educational institution name
- CGPA, GPA, or academic grades
- City, location, or geographical information
- Any personal characteristics unrelated to technical skills and experience

The evaluator.py component then processes the LLM's JSON-structured response, ensuring the generated scores respect the constraints defined in the template. This architecture guarantees that every candidate evaluation operates under the same strict fairness parameters.

Summary

• Fairness constraints in the evaluation prompts are hard-coded in prompts/templates/resume_evaluation_criteria.jinja and enforce three types of neutrality: demographic, educational, and geographic.
• The template explicitly prohibits scoring based on personal identifiers, institutional names, grades, or location data.
• Evaluation is restricted to technical merit only, including skills, project impact, open-source contributions, and production experience.
• The constraints are programmatically enforced at runtime by prompt.py (template rendering) and evaluator.py (LLM orchestration), ensuring consistent unbiased evaluation across all candidates.

Frequently Asked Questions

What specific personal attributes does the hiring-agent ignore during evaluation?

The system explicitly ignores the candidate's name, gender, age, ethnicity, and any personal demographic information. These fields are listed under the "SCORES MUST NEVER DEPEND ON" section in resume_evaluation_criteria.jinja, effectively preventing the LLM from incorporating protected characteristics into scoring calculations.

How does the system prevent bias from academic credentials?

The fairness constraints enforce educational neutrality by prohibiting the model from considering college or university names, as well as CGPA, GPA, or any academic grading metrics. This is defined in lines 7-10 of the criteria template, ensuring candidates are evaluated on demonstrated skills rather than institutional pedigree or grade point averages.

Which source files implement the fairness logic?

The primary fairness definitions reside in prompts/templates/resume_evaluation_criteria.jinja. Supporting implementation files include prompt.py, which loads and renders the template, and evaluator.py, which executes the LLM call and parses results. The README.md provides high-level documentation regarding the system's bias-mitigation goals.

Can the fairness constraints be customized or overridden?

The constraints are hard-coded directly into the Jinja template at prompts/templates/resume_evaluation_criteria.jinja. While the template itself could theoretically be modified since it is a file in the repository, the current implementation treats these fairness requirements as strict, non-configurable rules that are injected into every evaluation prompt by default.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}