Output Format of the Hiring Agent Evaluation Report: JSON Schema and Usage Guide

The Hiring Agent produces a structured JSON object that conforms to the EvaluationData Pydantic model, containing four category scores, bonus points, deductions, key strengths, and areas for improvement.

The interviewstreet/hiring-agent repository automates technical resume screening using Large Language Models (LLMs). Understanding the output format of the Hiring Agent evaluation report is essential for integrating this tool into existing HR pipelines, analytics dashboards, or custom hiring workflows. The report follows a strict, type-safe schema defined in Pydantic models, ensuring consistent structure across all candidate assessments.

Understanding the EvaluationData Schema

The core output structure is defined in models.py within the EvaluationData class (lines 44-50). This Pydantic model enforces a standardized schema that the LLM populates during the evaluation process, providing both quantitative metrics and qualitative assessments.

Top-Level Structure

The JSON object contains five primary fields:

• scores: An object containing four category assessments
• bonus_points: Total bonus points with a human-readable breakdown
• deductions: Total deduction points with narrative reasons
• key_strengths: A list of 1-5 candidate strengths
• areas_for_improvement: A list of 1-5 suggested development areas

Category Score Breakdown

Each of the four evaluation categories uses a CategoryScore sub-model defined in models.py:

• open_source: Contributions to open-source projects
• self_projects: Personal and side projects
• production: Production code experience
• technical_skills: Technical competency demonstration

Every category includes three fields: score (numeric rating), max (maximum possible score), and evidence (supporting details extracted from the resume).

How the Evaluation Report Is Generated

In evaluator.py, the ResumeEvaluator class orchestrates the evaluation process. Lines 75-86 handle the transformation from raw LLM output to structured data:

1. The LLM returns a JSON-encoded string
2. extract_json_from_response (from llm_utils.py) parses the raw text
3. The JSON validates against the EvaluationData schema
4. Returns a populated EvaluationData instance

Because the model inherits from Pydantic, you can serialize the output using model.json() for JSON strings or model.dict() for Python dictionaries.

Working with the Output Format

Serializing to JSON

To obtain a JSON string representation of the evaluation report:

from evaluator import ResumeEvaluator

evaluator = ResumeEvaluator()
evaluation = evaluator.evaluate_resume(resume_text)
json_report = evaluation.json()
print(json_report)

This produces a formatted JSON object:

{
  "scores": {
    "open_source": {"score": 8.5, "max": 10, "evidence": "..."},
    "self_projects": {"score": 7.0, "max": 10, "evidence": "..."},
    "production": {"score": 6.5, "max": 10, "evidence": "..."},
    "technical_skills": {"score": 9.0, "max": 10, "evidence": "..."}
  },
  "bonus_points": {"total": 12.0, "breakdown": "..."},
  "deductions": {"total": 3.0, "reasons": "..."},
  "key_strengths": ["Strong problem-solving", "Effective communication"],
  "areas_for_improvement": ["Depth of production experience", "Advanced tooling"]
}

Accessing Python Attributes

Access individual fields directly as Python attributes without parsing:

# Access specific category scores

open_source_score = evaluation.scores.open_source.score
max_possible = evaluation.scores.open_source.max
evidence_text = evaluation.scores.open_source.evidence

# Access bonus and deduction totals

bonus_total = evaluation.bonus_points.total
deduction_total = evaluation.deductions.total
reasons = evaluation.deductions.reasons

Converting to Dictionaries

For integration with data pipelines or persistent storage:

report_dict = evaluation.dict()

# Write to file for downstream processing

import json
with open("evaluation_report.json", "w") as f:
    json.dump(report_dict, f, indent=2)

Summary

• The Hiring Agent evaluation report follows the EvaluationData Pydantic schema defined in models.py (lines 44-50)
• Output includes four categorized scores (open_source, self_projects, production, technical_skills) with numeric values and evidence
• Bonus points and deductions include both numeric totals and narrative explanations
• The format supports JSON serialization via .json() and dictionary conversion via .dict()
• Lists of strengths and improvements contain 1-5 items each, as implemented in evaluator.py (lines 75-86)

Frequently Asked Questions

What data structure does the Hiring Agent return?

The Hiring Agent returns a Pydantic EvaluationData object that can be serialized to JSON or accessed as a Python class. This structure ensures type safety and includes nested models for scores, bonuses, and deductions, with validation enforced by the schema in models.py.

How are the evaluation scores categorized?

Scores are organized into four categories: open_source for contributions to public repositories, self_projects for personal work, production for professional experience, and technical_skills for general competencies. Each category contains a score, max value, and supporting evidence string.

Can I export the evaluation report to a file?

Yes. Use the .dict() method to convert the Pydantic model to a Python dictionary, then write it to a JSON file using the standard json module. Alternatively, use the .json() method to obtain a pre-formatted JSON string for direct file writing or API responses.

Where is the output format defined in the source code?

The schema is defined in models.py (lines 44-50) which specifies the EvaluationData class and its nested components like CategoryScore. The instantiation logic resides in evaluator.py (lines 75-86) where the LLM response transforms into the structured format using extract_json_from_response from llm_utils.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}