# How to Add New Evaluation Categories to the Hiring-Agent Resume Scorer

> Learn to add new evaluation categories to the Hiring-Agent Resume Scorer. Extend the Scores Pydantic model and update schema in models.py and resume_evaluation_criteria.jinja for custom scoring.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-06-27

---

**To add new evaluation categories beyond the default four in interviewstreet/hiring-agent, extend the `Scores` Pydantic model in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), update the JSON schema in `resume_evaluation_criteria.jinja`, and optionally adjust the CLI output in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) and CSV export in [`transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/transform.py).**

The interviewstreet/hiring-agent repository evaluates candidate resumes using four hard-coded scoring dimensions by default. When you need to assess additional competencies like leadership or system design, you must extend the schema and prompts to add new evaluation categories while maintaining the pipeline's data integrity.

## Understanding the Default Evaluation Structure

The hiring-agent scores resumes across four default dimensions defined in the **`Scores`** Pydantic model: `open_source`, `self_projects`, `production`, and `technical_skills`. These categories are hard-coded in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) and referenced throughout the evaluation pipeline, including the LLM prompt template, CSV transformer, and CLI pretty-printer. Because the iteration logic uses generic `model_dump().items()` calls, the system can dynamically handle additional fields without modifying the core scoring algorithm.

## Step-by-Step Guide to Adding Custom Evaluation Categories

To define new evaluation categories, update three critical areas of the codebase: the data schema, the LLM prompt, and the output formatters.

### Step 1: Extend the Scores Model in models.py

The **`Scores`** class in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) (lines 24-30) defines the schema that Pydantic validates. Add your new category as a **`CategoryScore`** field to include it in the `EvaluationData.scores` structure:

```python

# models.py

class Scores(BaseModel):
    open_source: CategoryScore
    self_projects: CategoryScore
    production: CategoryScore
    technical_skills: CategoryScore
    # New category – adjust the max value as you see fit

    leadership: CategoryScore

```

*Source:* [[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L24-L30)

### Step 2: Update the LLM Prompt Template

The **`resume_evaluation_criteria.jinja`** template instructs the LLM to return scores in a specific JSON structure. Add your category description and extend the JSON skeleton to ensure the model outputs the new field:

```jinja

## SCORING CRITERIA

### Leadership (0-15 points)

**HIGH SCORES (12-15 points):**
- Led a team of ≥3 engineers on a production-grade project.
- Demonstrated impact on product direction or organization growth.
- Organized open-source or community initiatives.

**MEDIUM SCORES (6-11 points):**
- Managed small teams or coordinated multi-person efforts.
- Showed mentorship or project-lead responsibilities.

**LOW SCORES (1-5 points):**
- Minor coordination tasks, no clear leadership impact.

...

# At the bottom, extend the JSON schema:

{
    "scores": {
        "open_source": {"score": 0, "max": 35, "evidence": "string"},
        "self_projects": {"score": 0, "max": 30, "evidence": "string"},
        "production": {"score": 0, "max": 25, "evidence": "string"},
        "technical_skills": {"score": 0, "max": 10, "evidence": "string"},
        "leadership": {"score": 0, "max": 15, "evidence": "string"}
    },
    ...
}

```

*Source:* [`prompts/templates/resume_evaluation_criteria.jinja`](https://github.com/interviewstreet/hiring-agent/blob/main/prompts/templates/resume_evaluation_criteria.jinja)

### Step 3: Verify CLI Output Handling in score.py

The **`print_evaluation_results`** function in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) iterates over `evaluation.scores.model_dump().items()`, meaning new categories appear automatically in the console output. No code changes are required unless you want custom formatting for specific fields:

```python

# score.py

def print_evaluation_results(evaluation: EvaluationData, candidate_name: str = "Candidate"):
    total_score = 0
    # Existing categories are printed in the loop below

    for cat, data in evaluation.scores.model_dump().items():
        print(f"🔹 {cat.replace('_', ' ').title()}: {data['score']}/{data['max']}")
        total_score += data['score']
    # The rest of the function stays unchanged

```

### Step 4: Ensure CSV Export Compatibility in transform.py

The **`transform_evaluation_response`** function in [`transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/transform.py) uses generic iteration to build CSV rows. New categories are automatically exported as `{category}_score`, `{category}_max`, and `{category}_evidence` columns without hard-coded field lists:

```python

# transform.py

def transform_evaluation_response(...):
    csv_row = {}
    if evaluation and hasattr(evaluation, "scores"):
        for cat, cat_data in evaluation.scores.model_dump().items():
            csv_row[f"{cat}_score"] = cat_data["score"]
            csv_row[f"{cat}_max"] = cat_data["max"]
            csv_row[f"{cat}_evidence"] = cat_data["evidence"]
    return csv_row

```

## Why the Generic Architecture Matters

The hiring-agent avoids hard-coded category lists in its processing logic. By using **`model_dump().items()`** to iterate over score fields, the code automatically accommodates schema extensions. This design means you only need to update the `Scores` definition and the prompt template to add new evaluation categories—no conditional logic or mapping tables require modification.

## Summary

- Extend the **`Scores`** model in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) with a new **`CategoryScore`** field to define the schema
- Update **`resume_evaluation_criteria.jinja`** to instruct the LLM on scoring criteria and JSON output format
- Verify that **[`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py)** and **[`transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/transform.py)** handle the new fields via their generic `model_dump().items()` iteration
- The default four categories (`open_source`, `self_projects`, `production`, `technical_skills`) serve as templates for new custom evaluation categories

## Frequently Asked Questions

### Where are the default four evaluation categories defined?

They are defined as fields in the **`Scores`** Pydantic model located in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) at lines 24-30. Each field uses the **`CategoryScore`** type to enforce score ranges and evidence requirements.

### Do I need to modify the core evaluation logic to add a new category?

No. The evaluation logic iterates generically over `evaluation.scores.model_dump().items()`, so it automatically processes any field present in the `Scores` model. You only need to update the schema, prompt template, and optionally the display formatting.

### What is the maximum score value for a new evaluation category?

The maximum value is defined in the **`CategoryScore`** field definition within the `Scores` model and specified in the JSON skeleton within `resume_evaluation_criteria.jinja`. You can set any integer value appropriate for your criteria, such as 15 points for leadership or 20 points for system design.

### Will adding a new category break existing CSV exports?

No. The **[`transform.py`](https://github.com/interviewstreet/hiring-agent/blob/main/transform.py)** module uses generic iteration to create CSV columns dynamically. When you add a field like `leadership`, the export automatically includes `leadership_score`, `leadership_max`, and `leadership_evidence` columns without requiring changes to the transformation logic.