How Resume Scores Are Calculated in Hiring Agent: LLM Evaluation and Capping Logic

Hiring Agent calculates resume scores by summing LLM-generated category scores with applied maximums, adding optional bonus points, subtracting deductions, and enforcing a final cap of 120 points (the sum of category maximums plus a 20-point buffer).

The interviewstreet/hiring-agent repository automates candidate evaluation through an LLM-based scoring system. Understanding how resume scores are calculated requires examining the aggregation logic in score.py that transforms raw model outputs into bounded numerical assessments.

The Resume Evaluation Pipeline

The process begins in evaluator.py, where the ResumeEvaluator.evaluate_resume method converts PDF resumes into structured text via transform.py, then prompts the LLM to return an EvaluationData object. This model—defined in models.py—contains the scores, bonus_points, and deductions fields that feed into the final calculation.

The data flows through this pipeline:

PDF → JSONResume → text conversion → LLM prompt → EvaluationData
          │                                          │
          └─→ optional GitHub / blog data ──────────┘

Category Scoring with Per-Category Caps

The scoring algorithm evaluates four distinct categories, each with a hard maximum defined in score.py (lines 78-85):

• open_source: 35 points maximum
• self_projects: 30 points maximum
• production: 25 points maximum
• technical_skills: 10 points maximum

The print_evaluation_results function processes these categories by capping each score before aggregation. According to lines 45-50 in score.py, the implementation uses:

total_score = 0
max_score = 0
for category_name, category_data in evaluation.scores.model_dump().items():
    capped_score = min(category_data["score"], category_data["max"])
    total_score += capped_score
    max_score += category_data["max"]

When displaying results, the system presents the capped value rather than the raw LLM output (lines 90-95), ensuring transparency regarding category limits.

Bonus Points and Deductions

After category summation, the algorithm applies optional adjustments. If bonus_points exists in the evaluation data, its total adds to the running score (lines 57-60 in score.py). Conversely, if deductions exist, their total subtracts from the score (lines 62-64).

# Apply bonuses

if evaluation.bonus_points:
    total_score += evaluation.bonus_points.total

# Apply deductions

if evaluation.deductions:
    total_score -= evaluation.deductions.total

These modifiers accommodate exceptional achievements or identified deficiencies outside standard categorical assessment.

The Overall Score Cap

The final step enforces an absolute maximum to prevent score inflation. The ceiling equals the sum of category maximums (100 points) plus a 20-point bonus buffer. Lines 65-70 in score.py implement this protection:

max_possible = max_score + 20  # 100 + 20 = 120

if total_score > max_possible:
    total_score = max_possible
    # Warning emitted when capping occurs

The complete calculation formula implemented in the repository:

def calculate_score(evaluation: EvaluationData) -> float:
    # 1. Category totals with caps

    total = 0
    max_score = 0
    for cat_name, cat in evaluation.scores.model_dump().items():
        capped = min(cat["score"], cat["max"])
        total += capped
        max_score += cat["max"]

    # 2. Add bonuses

    if evaluation.bonus_points:
        total += evaluation.bonus_points.total

    # 3. Subtract deductions

    if evaluation.deductions:
        total -= evaluation.deductions.total

    # 4. Enforce overall cap (category max sum + 20 bonus buffer)

    max_possible = max_score + 20
    return min(total, max_possible)

Summary

• Hiring Agent uses an LLM-based ResumeEvaluator in evaluator.py to generate initial scores stored in the EvaluationData model.
• Four categories contribute to the base score: open source (35), self projects (30), production (25), and technical skills (10).
• Per-category capping occurs at lines 45-50 in score.py, ensuring individual scores never exceed their defined maximums.
• Bonus points and deductions provide adjustable modifiers applied after category summation (lines 57-64).
• Overall cap limits the final score to 120 points (100 category maximums + 20 bonus buffer) as enforced at lines 65-70.

Frequently Asked Questions

What is the maximum possible resume score in Hiring Agent?

The absolute maximum is 120 points, comprising the 100-point category maximum sum plus a 20-point bonus buffer. The system enforces this cap in score.py (lines 65-70) to prevent scores from exceeding reasonable bounds, even when bonus points are awarded.

How does the LLM evaluation interact with the scoring algorithm?

The ResumeEvaluator.evaluate_resume method sends resume text (plus optional GitHub or blog data) to the LLM and parses the JSON response into an EvaluationData object defined in models.py. The score.py aggregation logic then processes this data, applying mathematical caps and adjustments to the raw LLM scores to produce the final result.

What happens if a category score exceeds its maximum?

If the LLM returns a score higher than the category limit (for example, 40 points for open source where the maximum is 35), the system silently caps the value during calculation using min(category_score, category_max) logic at lines 45-50 in score.py. The display logic (lines 90-95) shows the capped value to maintain consistency with the defined maximums.

Can bonus points push the score beyond the standard category maximums?

Yes, the 20-point bonus buffer explicitly allows the final score to exceed the 100-point category sum, accommodating exceptional achievements. However, the total cannot surpass 120 points due to the overall cap enforced immediately after bonus application in the scoring algorithm.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}