# How Scores Are Generated for Candidate Evaluations in Hiring Agent: A Technical Deep Dive

> Discover how Hiring Agent generates candidate evaluation scores. Learn about PDF parsing LLM assessments achievement categories bonus points and deductions in this technical deep dive.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: deep-dive
- Published: 2026-07-11

---

**Hiring Agent generates candidate evaluation scores by parsing PDF résumés into structured data, prompting an LLM to assess four achievement categories, then aggregating capped category scores with bonus points and deductions to produce a final clamped numeric result.**

Hiring Agent is an open-source automated résumé evaluator from InterviewStreet that uses Large Language Models (LLMs) to screen engineering candidates. Understanding exactly how it calculates evaluation scores enables teams to customize scoring weights and interpret candidate rankings accurately.

## Three-Stage Evaluation Pipeline

The scoring system operates through a deterministic pipeline that transforms raw PDF documents into structured numeric ratings.

### Stage 1: PDF Parsing and Enrichment

The process begins in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) where the `PDFHandler` class extracts structured data from candidate résumés.

```python
from pdf import PDFHandler

pdf_handler = PDFHandler()
resume_data = pdf_handler.extract_json_from_pdf(pdf_path)

```

This produces a `JSONResume` Pydantic model containing sections like `basics`, `work`, `education`, `skills`, and `projects`. If the candidate includes a GitHub profile, the system optionally enriches the data by fetching public repositories via `fetch_and_display_github_info` in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py), merging additional context into the résumé text before evaluation.

### Stage 2: LLM-Driven Assessment

The core scoring logic resides in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py). The `ResumeEvaluator.evaluate_resume` method (lines 48-87) constructs a detailed prompt using the *resume_evaluation_criteria* template and sends it to the configured LLM provider (Ollama or Gemini).

```python
from evaluator import ResumeEvaluator

evaluator = ResumeEvaluator()
evaluation = evaluator.evaluate_resume(resume_text)  # Returns EvaluationData

```

The LLM receives the complete résumé text plus a JSON schema defining the expected output structure. It returns an `EvaluationData` object (defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), lines 18-50) containing four categorized scores:

- **Open Source** (`open_source`)
- **Self Projects** (`self_projects`)
- **Production Experience** (`production`)
- **Technical Skills** (`technical_skills`)

Each category is a `CategoryScore` containing a raw `score`, a `max` ceiling, and an `evidence` string justifying the rating.

### Stage 3: Score Aggregation and Clamping

Final score calculation occurs in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py), primarily within the `print_evaluation_results` function. The algorithm implements three critical safeguards to prevent score inflation:

1. **Per-category capping**: Each score is clamped to its configured maximum (e.g., Open Source capped at 35 points)
2. **Bonus and deduction handling**: Adds points from `BonusPoints` and subtracts `Deductions`
3. **Global maximum enforcement**: Clamps the total to `category_maxes + 20` (where 20 represents the maximum possible bonus points)

```python

# Logic derived from score.py lines 41-73 and 78-86

total_score = 0
max_score = 0

for cat, data in evaluation.scores.model_dump().items():
    cat_score = min(data["score"], data["max"])  # Enforce per-category cap

    total_score += cat_score
    max_score += data["max"]

# Apply bonuses and deductions

total_score += evaluation.bonus_points.total
total_score -= evaluation.deductions.total

# Global clamp

max_possible = max_score + 20
final_score = min(total_score, max_possible)

```

## Core Data Models

The scoring system relies on strict Pydantic models defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) (lines 18-50) to ensure type safety and structured LLM outputs.

**CategoryScore** defines individual rating components:

```python
class CategoryScore(BaseModel):
    score: float
    max: float
    evidence: str

```

**EvaluationData** serves as the top-level schema containing all scoring dimensions:

```python
class Scores(BaseModel):
    open_source: CategoryScore
    self_projects: CategoryScore
    production: CategoryScore
    technical_skills: CategoryScore

class EvaluationData(BaseModel):
    scores: Scores
    bonus_points: BonusPoints
    deductions: Deductions
    key_strengths: List[str]
    areas_for_improvement: List[str]

```

## Customizing the Scoring Algorithm

Teams can modify scoring behavior by adjusting category maximums or bonus calculations in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py). The category maximums are defined at lines 80-85, while the aggregation logic resides in the `print_evaluation_results` function.

**Manual score aggregation** (mirroring the production implementation):

```python
def calculate_final_score(evaluation: EvaluationData) -> float:
    """Replicate Hiring Agent's scoring logic programmatically."""
    total, max_total = 0.0, 0.0
    
    for cat in evaluation.scores.model_dump().values():
        total += min(cat["score"], cat["max"])
        max_total += cat["max"]
    
    total += evaluation.bonus_points.total
    total -= evaluation.deductions.total
    
    return min(total, max_total + 20)  # 20 = MAX_BONUS_POINTS

```

## Running Evaluations

Execute the full pipeline from the command line:

```bash
python score.py path/to/candidate_resume.pdf

```

This invokes the `main()` function in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py), which orchestrates PDF parsing, optional GitHub enrichment, LLM evaluation, and final score computation.

## Summary

- **Hiring Agent** uses a three-stage pipeline: PDF parsing → LLM evaluation → score aggregation.
- **Per-category caps** prevent individual dimensions from exceeding predefined limits (e.g., 35 points for Open Source).
- **Bonus points** (maximum 20) and **deductions** adjust the final total before clamping to the global maximum.
- **Pydantic models** (`EvaluationData`, `CategoryScore`) enforce structured LLM outputs and type safety throughout the system.
- **Source files**: [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) handles LLM prompting, [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) manages aggregation logic, and [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) defines the data schemas.

## Frequently Asked Questions

### How does Hiring Agent prevent LLMs from generating inflated scores?

The system implements **strict output validation** using Pydantic schemas defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). Even if the LLM returns high raw values, the aggregation logic in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) enforces per-category maximums and a global cap of `category_maxes + 20`, ensuring scores remain within configured boundaries.

### Can I modify the category weights or maximum point values?

Yes. Edit the category maximums defined in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) (lines 80-85) and adjust the `MAX_BONUS_POINTS` constant (set to 20) to change the total score range. The system automatically recalculates the global clamp based on these configuration values.

### What happens if a candidate has no GitHub profile?

The evaluation proceeds normally. The `find_profile` function in [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) detects missing profiles, and the pipeline skips the GitHub enrichment step in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py). The LLM evaluates the candidate solely based on the PDF résumé content, though bonus points typically associated with open-source contributions may not apply.

### Which LLM providers does Hiring Agent support?

According to the source code in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), the system supports **Ollama** (for local models) and **Gemini** (Google's API). The `initialize_llm_provider` function selects the appropriate client based on environment configuration, allowing teams to run evaluations offline or via cloud APIs.