How to Customize Hiring Agent for Organization-Specific Hiring Requirements

You can customize Hiring Agent by modifying Jinja templates in prompts/templates/, extending the ResumeEvaluator class in evaluator.py, and adjusting scoring criteria to match your company's unique rubric without touching the core parsing logic.

Hiring Agent is a modular open-source pipeline from Interview Street that converts PDF resumes into structured evaluations enriched with GitHub data. Because each stage is isolated behind clear interfaces, you can adapt the system to match organization-specific hiring requirements while keeping the core logic stable. The modular architecture uses templates for extraction, Pydantic models for data validation, and a pluggable scoring engine that makes customization straightforward.

Understanding the Hiring Agent Architecture

The pipeline processes resumes through distinct stages, each defined in specific source files. Understanding this flow helps you identify exactly where to inject custom logic.

• PDF Parsing: pymupdf_rag.py converts PDF pages to Markdown using PyMuPDF
• Section Extraction: pdf.py and prompts/template_manager.py handle LLM-based section extraction using Jinja templates
• GitHub Enrichment: github.py fetches profile and repository data when usernames are detected
• Evaluation: evaluator.py generates fairness-aware scores using the EvaluationData Pydantic model from models.py
• Orchestration: score.py runs the complete pipeline and outputs CSV results when DEVELOPMENT_MODE=True

The config.py file and .env.example control provider selection and model configuration, while prompts/templates/ contains the LLM prompts that determine extraction behavior.

Customizing Resume Section Extraction

To capture organization-specific data points like "Leadership Experience" or "Certifications," you add new Jinja templates and wire them into the extraction pipeline.

Adding New Jinja Templates

Create a new template file under prompts/templates/. For example, leadership.jinja might contain:

You are a resume parsing assistant. Extract the candidate's leadership experience from the following markdown text. Return a JSON array of objects each containing:
- "title": the leadership role,
- "organisation": the organisation name,
- "duration": start‑month/year to end‑month/year (or "present"),
- "description": a concise bullet‑point list of responsibilities.

Only output valid JSON. If no leadership section is present, return an empty array.

Registering Templates in TemplateManager

Edit prompts/template_manager.py to register your new template in the _load_templates() method:

# prompts/template_manager.py

# Add an entry to the template_files dict:

"leadership": "leadership.jinja",

The TemplateManager now exposes render_template("leadership", text_content=md) for use in the extraction pipeline.

Integrating Custom Sections into the Pipeline

Modify pdf.py to call your new template and parse the results:

# pdf.py (excerpt)

leadership_json = self.provider.chat(
    model=self.model_name,
    messages=[
        {"role": "system", "content": self.system_message},
        {"role": "user", "content": self.template_manager.render_template(
            "leadership", text_content=markdown_text)},
    ],
    format=YourLeadershipSchema.model_json_schema(),
)

Define YourLeadershipSchema as a Pydantic model in models.py and merge it into the final JSONResume structure.

Modifying Evaluation Criteria and Scoring

The evaluation stage uses the resume_evaluation_criteria.jinja template to define categories, weightings, and fairness constraints. You can customize this to reflect your organization's priorities.

Editing the Evaluation Template

Modify prompts/templates/resume_evaluation_criteria.jinja to adjust category weights or add new criteria like "Culture Fit" or "Domain Expertise." The template defines how the LLM assigns scores across dimensions such as Open Source Contributions, Production Experience, and Technical Skills.

Adjusting Scoring Constants

For hard limits on scoring ranges, edit the constants in evaluator.py:

# evaluator.py

MAX_BONUS_POINTS = 30          # increase total bonus pool

MIN_FINAL_SCORE = -10          # raise the floor

MAX_FINAL_SCORE = 150          # allow higher peaks

These constants control the fairness boundaries applied to all candidate evaluations.

Implementing Custom Evaluator Logic

For complex organization-specific rules (like mandatory certifications or minimum tenure requirements), extend the ResumeEvaluator class:

# custom_evaluator.py

from evaluator import ResumeEvaluator
from models import EvaluationData

class OrgEvaluator(ResumeEvaluator):
    def evaluate_resume(self, resume_text: str) -> EvaluationData:
        base_data = super().evaluate_resume(resume_text)

        # Example: enforce minimum certification requirements

        if "certifications" in resume_text.lower():
            base_data.bonus_points += 5
        else:
            base_data.deduction_points += 5

        # Recompute final score according to organization policy

        base_data.final_score = (
            base_data.open_source + base_data.self_projects +
            base_data.production + base_data.technical_skills +
            base_data.bonus_points - base_data.deduction_points
        )
        return base_data

Modify score.py to instantiate OrgEvaluator instead of the base ResumeEvaluator when running the pipeline.

Complete Implementation Example

Here is a minimal implementation that ties together custom templates and evaluation logic:

# custom_run.py

import os
from score import ScoreRunner
from custom_evaluator import OrgEvaluator

if __name__ == "__main__":
    pdf_path = "samples/engineer_resume.pdf"
    
    # Initialize with organization-specific evaluator

    evaluator = OrgEvaluator()
    runner = ScoreRunner(evaluator=evaluator)
    
    # Run the full pipeline: PDF → GitHub → Custom Evaluation

    runner.run(pdf_path)

Running python custom_run.py processes the resume through your custom extraction templates, applies the GitHub enrichment from github.py, and scores according to your organization's rubric defined in OrgEvaluator.

Summary

• Template customization: Add Jinja files to prompts/templates/ and register them in TemplateManager._load_templates to extract new resume sections.
• Scoring adjustments: Edit resume_evaluation_criteria.jinja and modify constants in evaluator.py to match your organization's weightings and bounds.
• Complex logic: Subclass ResumeEvaluator to implement mandatory rules like certification requirements or tenure thresholds.
• Data models: Extend Pydantic models in models.py to support new fields extracted from custom templates.
• Pipeline integration: Update score.py to use your custom evaluator subclass for production runs.

Frequently Asked Questions

How do I add a completely new evaluation category to the scoring rubric?

Edit the resume_evaluation_criteria.jinja template to include your new category with specific evaluation criteria and weighting instructions. Add corresponding fields to the EvaluationData Pydantic model in models.py to store the scores, and update the final score calculation in evaluator.py or your custom subclass to include the new category in the total.

Can I disable the GitHub enrichment for internal candidates who may not have public repositories?

Yes. The GitHub enrichment in github.py is optional and can be bypassed by modifying the pipeline in score.py. You can also set environment variables in .env to control whether the enrichment step runs, or simply extend the ResumeEvaluator to skip GitHub-dependent scoring criteria when profile data is unavailable.

What is the best way to enforce hard requirements like mandatory certifications?

Create a subclass of ResumeEvaluator in a new file (e.g., custom_evaluator.py) and override the evaluate_resume method. Use this method to check for specific keywords or patterns in the resume text before calling the parent class logic, and add deduction points or raise exceptions for candidates who fail to meet minimum requirements. Instantiate this subclass in score.py instead of the base evaluator.

How do I test my customizations without running the full pipeline?

Set DEVELOPMENT_MODE=True in your environment or config.py to enable CSV export debugging. You can also unit test individual components by importing TemplateManager to verify template rendering, or instantiate your custom ResumeEvaluator with sample resume text to validate scoring logic before integrating with the PDF parsing and GitHub enrichment stages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}