What Are the Four Score Categories in Hiring Agent Evaluations?

Hiring Agent evaluates résumés across four distinct categories: Open Source (35 points), Self Projects (30 points), Production Experience (25 points), and Technical Skills (10 points).

The interviewstreet/hiring-agent repository implements a structured scoring rubric to assess candidate qualifications programmatically. Understanding these Hiring Agent evaluation score categories helps developers interpret how their contributions, experience, and technical abilities are weighted by the automated evaluation engine.

The Four Hiring Agent Evaluation Score Categories

The evaluation logic assigns points across four specific domains defined in the Scores model. Each category has a predetermined maximum point value that contributes to the final assessment.

Open Source (35 points)

The Open Source category measures contributions to public open-source projects, including merged pull requests, maintained repositories, and community involvement. With a maximum of 35 points, this category carries the highest weight in the evaluation, reflecting the repository's emphasis on public code quality and collaborative development practices.

Self Projects (30 points)

The Self Projects category assesses personal side-projects, hobby work, and independent codebases that demonstrate initiative and technical curiosity. This category allows candidates to showcase work outside of professional obligations, awarding up to 30 points for demonstrable personal technical accomplishments.

Production Experience (25 points)

The Production Experience category evaluates professional, production-grade work history, including employed roles, shipped features, and enterprise-level system contributions. Worth up to 25 points, this category validates the candidate's ability to operate within professional software development lifecycles and maintain production systems.

Technical Skills (10 points)

The Technical Skills category examines the breadth and depth of relevant technical proficiencies, including programming languages, frameworks, and tooling expertise. At 10 points, this category has the lowest individual weight but serves to round out the technical assessment by verifying claimed competencies.

How Scores Are Structured in the Source Code

According to the interviewstreet/hiring-agent source code, these categories are formally defined in models.py (lines 24-28) within the Scores model. The model aggregates four CategoryScore objects, each containing a score field for the actual points awarded, a max field defining the category ceiling, and an evidence string that documents the reasoning behind the specific score.

The maximum point values referenced above are encoded in score.py (lines 81-85), where the evaluation results are calculated and printed. The final score computation sums the capped category scores (ensuring no category exceeds its maximum), adds any bonus points, and subtracts applicable deductions.

Working with Evaluation Scores Programmatically

You can access individual category scores from an EvaluationData object returned by the evaluation engine:

from models import EvaluationData

def show_category_scores(evaluation: EvaluationData):
    # Access each category directly from the Scores model

    print("Open Source score:", evaluation.scores.open_source.score)
    print("Self Projects score:", evaluation.scores.self_projects.score)
    print("Production Experience score:", evaluation.scores.production.score)
    print("Technical Skills score:", evaluation.scores.technical_skills.score)

# Example usage after running the evaluator

evaluation = main("example_resume.pdf")  # returns an EvaluationData object

show_category_scores(evaluation)

For reporting or API integration, convert the scores to a dictionary format:

def scores_to_dict(evaluation: EvaluationData) -> dict:
    return {
        "open_source": evaluation.scores.open_source.model_dump(),
        "self_projects": evaluation.scores.self_projects.model_dump(),
        "production": evaluation.scores.production.model_dump(),
        "technical_skills": evaluation.scores.technical_skills.model_dump(),
    }

Summary

• Open Source (35 points): Measures public contributions and collaborative development activity.
• Self Projects (30 points): Evaluates personal technical initiatives and independent work.
• Production Experience (25 points): Assesses professional, employment-based software development.
• Technical Skills (10 points): Verifies proficiency with specific languages, tools, and frameworks.
• The Scores model in models.py encapsulates these four categories as CategoryScore objects with score, max, and evidence attributes.
• Point calculations occur in score.py, where category scores are capped at their defined maximums before final summation.

Frequently Asked Questions

What is the highest-weighted category in Hiring Agent evaluations?

The Open Source category carries the highest weight at 35 points, reflecting the evaluation's emphasis on public code contributions and collaborative development experience over other qualification types.

How are the four score categories defined in the Hiring Agent source code?

The categories are defined in the Scores model within models.py (lines 24-28), which instantiates four CategoryScore objects named open_source, self_projects, production, and technical_skills. Each object tracks the awarded points, maximum allowable points, and supporting evidence.

Can individual category scores exceed their maximum values?

No. The evaluation logic explicitly caps category scores at their defined maximums (35, 30, 25, and 10 points respectively) during the final calculation phase in score.py. The CategoryScore objects store both the raw assessment and the capped values used in the final summation.

Where are the point values for each category configured?

The maximum point values for each of the four categories are defined in score.py (lines 81-85), where the print_evaluation_results function references these constants when displaying and calculating the final evaluation metrics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}