# How Evaluation Data Validation Works with Pydantic in the Hiring-Agent Repository

> Learn how the hiring-agent repository uses Pydantic for robust evaluation data validation, ensuring type safety and data integrity for scores, bonus points, and key strengths.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: internals
- Published: 2026-07-14

---

**The hiring-agent repository validates evaluation results using a strict Pydantic `EvaluationData` model that automatically enforces type safety, numeric constraints, and collection size limits on fields like `scores`, `bonus_points`, and `key_strengths`.**

The interviewstreet/hiring-agent repository implements rigorous evaluation data validation to ensure that candidate assessments meet strict structural requirements before processing. By defining a comprehensive `EvaluationData` schema in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py), the application leverages Pydantic to automatically validate incoming data from LLM responses or external services. This validation layer acts as a gatekeeper that prevents malformed evaluation data from propagating through the hiring pipeline.

## The EvaluationData Model Architecture

In [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py), the `EvaluationData` class serves as the root validation schema for all evaluation results. This Pydantic model defines the exact structure expected for candidate assessments, utilizing nested models to validate complex hierarchical data.

### Nested Model Definitions

The `EvaluationData` model relies on several specialized Pydantic models to validate specific components:

- **`Scores`**: Contains four required `CategoryScore` fields (`open_source`, `self_projects`, `production`, `technical_skills`)
- **`CategoryScore`**: Validates individual score entries with numeric bounds and evidence strings
- **`BonusPoints`**: Enforces non-negative totals with a maximum cap and required breakdown descriptions
- **`Deductions`**: Ensures deduction totals are non-negative and accompanied by explanatory reasons

## Validation Constraints and Business Rules

Pydantic automatically validates each field against Python type hints and constraints declared with the `Field` function. The hiring-agent repository implements specific business logic through these constraint definitions.

### Numeric Range Validation

The `bonus_points` field enforces a `total` value between 0 and 20 using `ge=0` and `le=20` constraints. Similarly, the `deductions` model requires `total` ≥ 0. These validations ensure that final scores remain within acceptable business ranges and prevent negative adjustments from entering the system.

### Collection Size Limits

For qualitative feedback, both `key_strengths` and `areas_for_improvement` are validated as `List[str]` with `min_items=1` and `max_items=5`. This prevents empty feedback submissions while keeping responses concise and focused on the most critical points.

## Validation in Practice

When the application receives evaluation data—typically from an LLM response or external service—it instantiates the `EvaluationData` model directly from the raw JSON payload.

### Valid Data Example

```python
from main.models import EvaluationData, Scores, CategoryScore, BonusPoints, Deductions

# Example of building a valid evaluation payload

payload = {
    "scores": {
        "open_source": {"score": 8.5, "max": 10, "evidence": "Contributed to 3 repos"},
        "self_projects": {"score": 7.0, "max": 10, "evidence": "Built a personal API"},
        "production": {"score": 9.0, "max": 10, "evidence": "Deployed at scale"},
        "technical_skills": {"score": 8.0, "max": 10, "evidence": "Strong Python/SQL"},
    },
    "bonus_points": {"total": 5.0, "breakdown": "Open‑source contributions"},
    "deductions": {"total": 2.0, "reasons": "Minor style issues"},
    "key_strengths": ["Problem solving", "Team collaboration"],
    "areas_for_improvement": ["Testing coverage", "Documentation"],
}

# Pydantic validates on instantiation; raises ValidationError on bad data

evaluation = EvaluationData(**payload)
print(evaluation.json(indent=2))

```

### Handling ValidationError Exceptions

When validation fails, Pydantic raises a `ValidationError` that captures all constraint violations. The calling code in [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py) can catch these exceptions to prevent malformed data from being stored or processed further.

```python
from pydantic import ValidationError
from main.models import EvaluationData

bad_payload = {
    "scores": {},                     # missing required categories -> ValidationError

    "bonus_points": {"total": -1},    # total < 0 violates ge=0 -> ValidationError

    "deductions": {"total": -5},
    "key_strengths": [],              # empty list fails min_items=1 -> ValidationError

    "areas_for_improvement": ["A"] * 6,  # 6 items exceeds max_items=5 -> ValidationError

}

try:
    EvaluationData(**bad_payload)
except ValidationError as e:
    print(e)

```

## Integration with the Evaluation Pipeline

The [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py) module consumes validated `EvaluationData` instances when scoring candidates. By relying on Pydantic's validation guarantees, the evaluator can safely access nested attributes without defensive type checking, knowing that all `CategoryScore` objects contain valid numeric scores and required evidence strings.

## Summary

- The `EvaluationData` model in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py) defines the complete schema for evaluation data validation using Pydantic
- **Field constraints** using `ge`, `le`, `min_items`, and `max_items` enforce business rules on numeric ranges and list sizes
- **Nested models** (`Scores`, `BonusPoints`, `Deductions`) validate complex hierarchical data structures with strict type checking
- Pydantic raises `ValidationError` immediately on instantiation, preventing malformed evaluation data from entering the processing pipeline
- The [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py) module relies on these validation guarantees to safely process candidate assessments without additional defensive coding

## Frequently Asked Questions

### What triggers the validation in the hiring-agent repository?

Validation occurs when the `EvaluationData` model is instantiated from raw data, typically when receiving JSON responses from LLMs or external evaluation services. Pydantic checks all fields, types, and constraints during this initialization process according to the definitions in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py). Any violation immediately raises a `ValidationError` before the data reaches business logic.

### How does the validation prevent invalid bonus points from being accepted?

The `BonusPoints` model uses `Field(ge=0, le=20)` to constrain the `total` field. If a payload contains a negative value or exceeds 20 points, Pydantic raises a `ValidationError` immediately, preventing the invalid evaluation data from reaching the scoring logic in [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py).

### Can the validation handle partial evaluation data or missing fields?

No, the `EvaluationData` model requires all specified fields including the four category scores in the `scores` object. Missing required fields trigger validation errors. This strictness ensures that [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py) always receives complete assessment data with all required `CategoryScore` entries present.

### What happens when list constraints are violated for strengths or improvements?

If `key_strengths` or `areas_for_improvement` contain fewer than one item or more than five items, Pydantic raises a `ValidationError` citing the `min_items` or `max_items` constraint violation. This enforces concise feedback while ensuring at least one strength and improvement area is always documented.