# What Metrics Does evaluator.py Provide for Candidate Performance?

> Discover the candidate performance metrics provided by evaluator.py. Learn about category scores, bonus points, deductions, strengths, and areas for improvement for better hiring decisions.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: api-reference
- Published: 2026-07-15

---

**The [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) module in the interviewstreet/hiring-agent repository provides six categories of performance metrics: category scores (open-source, self-projects, production, technical skills), bonus points, deductions, key strengths, areas for improvement, and score boundaries, all structured within an `EvaluationData` Pydantic model.**

The [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) file serves as the core assessment engine in the interviewstreet/hiring-agent project, orchestrating LLM-based resume evaluations. It transforms unstructured resume text into quantifiable performance data by parsing structured responses into the `EvaluationData` schema defined in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py).

## How evaluator.py Structures Candidate Assessments

The evaluation workflow centers on the `ResumeEvaluator` class in [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py), which sends resume content to a language model and returns a structured `EvaluationData` object. This object encapsulates both quantitative scoring and qualitative feedback, enabling downstream ranking and automated decision-making.

The metrics system operates with defined mathematical boundaries. According to the source code in [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py), the system enforces:

- `MAX_FINAL_SCORE = 120`
- `MIN_FINAL_SCORE = -20`
- `MAX_BONUS_POINTS = 20`

## Detailed Performance Metrics Breakdown

### Category Scores (Technical Competency)

The `scores` attribute within `EvaluationData` contains four distinct category scores defined in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py):

- **open_source**: Evaluates contributions to open-source projects
- **self_projects**: Assesses independent project work
- **production**: Measures production-level system experience
- **technical_skills**: Rates overall technical capability

Each category includes:

- `score`: Numeric points achieved
- `max`: Maximum possible points for that category
- `evidence`: Textual explanation supporting the rating

### Bonus Points and Deductions (Adjustment Factors)

Beyond base category scores, [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) captures adjustment factors through two distinct fields:

**Bonus Points** (`bonus_points`):

- `total`: Aggregated extra points (capped at 20)
- `breakdown`: Textual description of outstanding resume aspects justifying the bonus

**Deductions** (`deductions`):

- `total`: Positive value subtracted from the final score
- `reasons`: Detailed explanation of missing or weak elements triggering penalties

### Qualitative Feedback Dimensions

The evaluation captures narrative insights through two list-based fields defined in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py):

**Key Strengths** (`key_strengths`): A curated list of 1–5 standout qualifications identified by the model.

**Areas for Improvement** (`areas_for_improvement`): A targeted list of 1–5 development opportunities or gaps detected in the resume.

### Score Boundaries and Validation Constants

The [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py) file defines hard constraints that govern all calculations:

- Final scores cannot exceed **120 points**
- Final scores cannot drop below **-20 points**
- Bonus points are strictly limited to **20 points maximum**

These constants ensure consistent evaluation ranges across all candidate assessments.

## Accessing Metrics Programmatically

To retrieve these metrics, instantiate `ResumeEvaluator` and call `evaluate_resume()`:

```python
from evaluator import ResumeEvaluator

# Initialise the evaluator (defaults to the configured LLM model)

evaluator = ResumeEvaluator()

# Provide a plain-text resume

resume_text = """John Doe
Software Engineer with 5 years of experience...
"""

# Run the evaluation

data = evaluator.evaluate_resume(resume_text)

# Access category scores

print("Open-source score:", data.scores.open_source.score, "/", data.scores.open_source.max)
print("Technical-skills score:", data.scores.technical_skills.score)

# Access adjustment factors

print("Bonus points:", data.bonus_points.total)
print("Deductions:", data.deductions.total)

# Access qualitative feedback

print("Key strengths:", data.key_strengths)
print("Improvement areas:", data.areas_for_improvement)

```

The `EvaluationData` Pydantic model validates all fields automatically, ensuring type safety for the `scores`, `bonus_points`, and `deductions` nested objects.

## Summary

- **[`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)** drives the resume assessment workflow in interviewstreet/hiring-agent by interfacing with language models and structuring responses into quantifiable metrics.
- **Six metric categories** comprise the evaluation: category scores (four technical dimensions), bonus points, deductions, key strengths, areas for improvement, and enforced score boundaries.
- **Score ranges** are strictly bounded between -20 and 120, with bonus points capped at 20.
- **Structured schemas** in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py) define `EvaluationData`, `Scores`, `BonusPoints`, and `Deductions` to ensure consistent data validation.
- **Programmatic access** occurs through the `ResumeEvaluator.evaluate_resume()` method, returning a fully populated `EvaluationData` instance.

## Frequently Asked Questions

### How is the final candidate score calculated from the individual metrics?

The final score aggregates the four category scores from the `scores` object, adds the `bonus_points.total`, and subtracts the `deductions.total`. According to [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py), this calculated value must fall within the `MIN_FINAL_SCORE` (-20) and `MAX_FINAL_SCORE` (120) constants, though the specific aggregation logic may apply these bounds as constraints.

### What is the difference between category scores and bonus points in evaluator.py?

**Category scores** (`scores`) evaluate specific technical competencies (open-source, self-projects, production, technical skills) against defined maximums, while **bonus points** recognize exceptional achievements outside standard criteria. Deductions function similarly to negative bonus points, penalizing missing elements. This separation allows hiring teams to distinguish between baseline competency and exceptionalism.

### Where are the performance metric schemas defined in the hiring-agent repository?

All Pydantic schemas reside in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py). The `EvaluationData` class serves as the top-level container, containing nested `Scores`, `BonusPoints`, and `Deductions` models. Each schema field includes type validation and documentation, with `key_strengths` and `areas_for_improvement` defined as constrained lists in the `EvaluationData` definition.

### What are the minimum and maximum possible scores in the evaluation system?

The [`main/evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/evaluator.py) file explicitly defines `MIN_FINAL_SCORE = -20` and `MAX_FINAL_SCORE = 120`, creating a 140-point dynamic range. Additionally, the `MAX_BONUS_POINTS = 20` constant limits discretionary positive adjustments. These boundaries ensure evaluation consistency regardless of resume quality extremes.