Resume Evaluator Scoring Categories and Maximum Points in Hiring Agent

The Hiring Agent resume evaluator scores candidates across four distinct categories—Open Source (35 points), Self Projects (30 points), Production (25 points), and Technical Skills (10 points)—with strict maximum limits enforced via the evaluation criteria template.

The interviewstreet/hiring-agent repository implements a structured resume evaluation system that assigns quantitative scores based on specific dimensions of a candidate's experience. These scoring boundaries, defined in the resume evaluation criteria template, ensure consistent assessments while preventing any single dimension from exceeding its designated weight.

The Four Scoring Categories

The evaluator categorizes resume content into four distinct scoring dimensions. Each category carries a specific maximum point value as defined in prompts/templates/resume_evaluation_criteria.jinja at lines 58-62.

Open Source Contributions (35 Points)

Open Source carries the highest weight in the evaluation framework, allowing up to 35 points for demonstrated contributions to public repositories. This category recognizes collaborative development experience, code review participation, and community engagement as primary indicators of technical capability.

Self Projects (30 Points)

Self Projects can contribute up to 30 points, rewarding candidates who build independent applications, tools, or experimental code outside professional obligations. This dimension evaluates initiative, practical coding skills, and the ability to architect and complete projects autonomously.

Production Experience (25 Points)

Production experience is capped at 25 points, covering professional work in live, deployed environments. This category assesses exposure to enterprise-grade systems, operational responsibilities, and real-world maintenance of critical infrastructure.

Technical Skills (10 Points)

Technical Skills has the lowest ceiling at 10 points, focusing on specific technologies, programming languages, and frameworks listed in the resume. This dimension provides a baseline measurement of tool familiarity without allowing tool shopping to dominate the overall evaluation.

How Maximum Points Are Enforced

The scoring limits are hardcoded in the resume evaluation criteria template to ensure the LLM-based evaluator never assigns points beyond the defined thresholds. According to the source code in prompts/templates/resume_evaluation_criteria.jinja, these constraints are explicitly listed in the scoring instructions provided to the language model.

The evaluator.py module consumes this template to construct evaluation prompts, ensuring that every resume assessment adheres to the four-category framework with the specified maximum values. This implementation prevents score inflation and maintains evaluation consistency across different candidate profiles.

Code Implementation Reference

When integrating with the scoring system programmatically, the maximum values can be referenced using the category mapping defined in the evaluation logic:

# Maximum points for each scoring category

CATEGORY_MAX = {
    "open_source": 35,
    "self_projects": 30,
    "production": 25,
    "technical_skills": 10,
}

This dictionary structure mirrors the scoring constraints enforced by the LLM prompt template, allowing developers to validate scores programmatically or integrate the limits into custom evaluation pipelines.

Summary

• The Hiring Agent evaluator uses four scoring categories with distinct maximum values totaling 100 points: Open Source (35), Self Projects (30), Production (25), and Technical Skills (10).
• Score limits are defined in prompts/templates/resume_evaluation_criteria.jinja at lines 58-62 and enforced during LLM prompt generation in evaluator.py.
• Open Source contributions receive the highest weight (35 points), while Technical Skills receives the lowest (10 points), prioritizing demonstrated practical experience over tool familiarity.
• The system prevents category scores from exceeding their maximums through explicit prompt instructions to the language model.

Frequently Asked Questions

What is the highest scoring category in the Hiring Agent evaluator?

Open Source is the highest weighted category with a maximum of 35 points, followed by Self Projects at 30 points. This weighting prioritizes collaborative development and public code contributions as stronger indicators of candidate quality than formal technical skills listings or tooling knowledge.

Where are the scoring category limits defined in the codebase?

The maximum point values are explicitly defined in prompts/templates/resume_evaluation_criteria.jinja between lines 58-62. This template file serves as the authoritative source for scoring constraints, which are then processed by evaluator.py to generate evaluation prompts for the LLM.

Can individual category scores exceed their maximum points?

No, the system enforces strict caps through the evaluation criteria template provided to the LLM. The prompt instructions explicitly constrain the model to never assign scores above the defined limits (35, 30, 25, and 10 respectively), ensuring consistent evaluation standards across all resumes processed by the hiring agent.

How are the scoring categories implemented in the evaluation pipeline?

The evaluator.py module loads the resume_evaluation_criteria.jinja template and passes it to the language model along with the resume content. The template contains specific scoring rubric instructions for each category's maximum points, guiding the LLM to produce normalized scores that respect the defined boundaries while assessing candidate qualifications across the four dimensions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}