# Fairness Constraints in Hiring Agent's Evaluation: Technical Implementation Guide

> Learn how Hiring Agent implements fairness constraints. Discover prompt engineering for LLMs, bias prevention, merit scoring, and Pydantic validation for fair hiring.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-11

---

**Hiring Agent enforces strict fairness constraints through prompt-engineered LLM instructions that prohibit demographic bias, mandate technical merit scoring, enforce category point caps, and validate structured output via Pydantic schemas.**

The `interviewstreet/hiring-agent` repository implements a bias-resistant resume evaluation system that relies on explicit fairness constraints embedded in LLM prompts. These constraints ensure that candidate assessments depend solely on technical ability rather than demographic characteristics. Understanding how these safeguards are implemented helps developers audit and maintain equitable AI-driven hiring tools.

## Core Fairness Rules in Resume Evaluation

### Demographic Blindness Requirements

The evaluation criteria explicitly prohibit the LLM from considering personal characteristics. According to the `resume_evaluation_criteria.jinja` template, scores must never depend on a candidate's name, gender, age, ethnicity, college name, GPA, city, or any attribute unrelated to technical capability.

### Technical Merit Scoring Focus

Evaluations must derive solely from six specific technical areas:

- Technical skills and programming languages
- Project complexity and real-world impact
- Open-source contributions and community involvement
- Production-level work experience
- Technical communication and documentation
- Problem-solving demonstrated in projects

### Program Distinction Safeguards

The system treats "Google Summer of Code (GSoC)" and "Girl Script Summer of Code" as distinct programs within the prompt template. This prevents scoring shortcuts that could create bias between similarly structured programs with different names.

### Hard Score Limits and Ceilings

The `EvaluationData` schema in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) enforces strict numeric boundaries:

- **Open-source**: 0-35 points
- **Self-projects**: 0-30 points
- **Production**: 0-25 points
- **Technical skills**: 0-10 points
- **Bonus points**: Maximum 20 total
- **Overall score**: Maximum 120 points (categories + bonus - deductions)

Additionally, every score field must contain non-empty evidence strings, and all category scores must be ≥ 0 with deductions captured separately.

## How Constraints Are Enforced at Runtime

### Prompt Construction

The `ResumeEvaluator._load_evaluation_prompt` method renders the Jinja template containing fairness rules. This prepares the system message that instructs the LLM on acceptable evaluation criteria.

### LLM Provider Interaction

The composed prompt is sent to the configured provider, either `models.OllamaProvider` or `models.GeminiProvider`, depending on deployment configuration.

### Response Validation

Raw LLM output passes through `extract_json_from_response` and undergoes strict validation against the `EvaluationData` Pydantic model. Any deviation from the schema—such as missing evidence fields or scores exceeding defined maximums—raises validation errors that halt processing.

### Post-Processing Guards

Validation failures force developers to adjust either the prompt engineering or the LLM behavior, creating a feedback loop that maintains constraint integrity across evaluations.

## Implementation in the Source Code

The fairness architecture spans multiple files in the repository:

- **`prompts/templates/resume_evaluation_criteria.jinja`**: Contains the explicit fairness rules and scoring criteria embedded in the LLM prompt.
- **[`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)**: Houses the `ResumeEvaluator` class with the `_load_evaluation_prompt` method that orchestrates prompt assembly.
- **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)**: Defines the `EvaluationData` Pydantic schema that enforces field presence and numeric constraints at runtime.
- **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) and [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)**: Provide provider selection and response parsing utilities.
- **[`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py)**: CLI entry point that orchestrates end-to-end execution from PDF ingestion to final evaluation.

## Practical Code Example

The following demonstrates how fairness constraints are automatically enforced during evaluation:

```python
from evaluator import ResumeEvaluator

# Load resume text (already extracted from PDF)

with open("sample_resume.md") as f:
    resume_text = f.read()

evaluator = ResumeEvaluator()                # Uses DEFAULT_MODEL from config

evaluation = evaluator.evaluate_resume(resume_text)

print(evaluation.json(indent=2))

```

The returned `evaluation` object contains:

- Four mandatory score categories respecting their maximum point caps
- A `bonus_points` object with `total` ≤ 20
- A `deductions` object for penalties
- Required evidence strings for every scored field
- `key_strengths` and `areas_for_improvement` limited to required counts

If the LLM attempts to assign demographic-based scores or exceed point limits, the `EvaluationData` validation raises an immediate error, preventing biased outputs from reaching downstream systems.

## Summary

- Hiring Agent's fairness constraints are embedded directly in the LLM prompt template at `prompts/templates/resume_evaluation_criteria.jinja`.
- Demographic characteristics including name, gender, age, ethnicity, and education institution are explicitly excluded from scoring considerations.
- Technical merit evaluation focuses on six specific competency areas with strict evidentiary requirements.
- Pydantic schema validation in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) enforces hard numeric ceilings on all score categories and bonus points.
- The four-stage pipeline (prompt construction, LLM invocation, JSON extraction, and schema validation) ensures constraints apply consistently across `OllamaProvider` and `GeminiProvider` backends.

## Frequently Asked Questions

### How does Hiring Agent prevent demographic bias in resume scoring?

The system embeds explicit prohibitions in the evaluation criteria template, instructing the LLM to ignore names, gender, age, ethnicity, college names, GPA, and geographic location. These instructions are enforced at the prompt level and validated through the `EvaluationData` schema, ensuring scores derive solely from demonstrated technical abilities.

### What are the maximum point allocations for each evaluation category?

According to the schema defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), the category limits are: Open-source (35 points), Self-projects (30 points), Production (25 points), and Technical skills (10 points). Bonus points cannot exceed 20, and the total combined score has a hard ceiling of 120 points.

### Which source files contain the fairness constraint implementation?

The constraints are defined in `prompts/templates/resume_evaluation_criteria.jinja`, loaded by `ResumeEvaluator._load_evaluation_prompt` in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), and enforced through the `EvaluationData` Pydantic model in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). Response parsing utilities in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) handle the extraction and initial formatting of LLM outputs before validation.

### What happens if the LLM generates scores that violate fairness constraints?

The `EvaluationData` schema validation catches violations such as missing evidence strings, scores exceeding category maximums, or negative values. These validation errors force immediate correction by the developer, either through prompt refinement or LLM behavior adjustment, preventing non-compliant evaluations from being processed.