How to Customize Hiring Workflows in Hiring-Agent: A Complete Guide

You customize hiring workflows in the interviewstreet/hiring-agent repository by editing Jinja templates in prompts/templates/, adjusting scoring constants in evaluator.py, and modifying the modular Python pipeline in files like pdf.py, github.py, and score.py.

The hiring-agent repository is an open-source, modular pipeline designed to automate resume screening and candidate evaluation. Because each major stage—PDF extraction, section parsing, GitHub enrichment, and scoring—is isolated in separate Python modules and driven by Jinja templates, you can tailor the hiring workflow without rewriting the entire application.

Understanding the Modular Pipeline Architecture

The hiring workflow follows a strict pipeline pattern where data flows through discrete stages. According to the source code in interviewstreet/hiring-agent, the process begins with PDF ingestion, moves through LLM-powered section extraction, adds external GitHub data, and concludes with structured evaluation. Each stage exposes specific extension points through template files and configuration constants.

Customizing Resume Data Extraction

Modifying PDF-to-Markdown Conversion

The initial text extraction happens in pymupdf_rag.py, which converts raw PDF pages into Markdown-like text. This is the first transformation step; any changes here alter the raw input that all downstream stages receive. The pdf.py module then orchestrates section-wise LLM calls, using the extracted text as context for prompts defined in Jinja templates.

To change how the system interprets resume formatting or handles specific PDF layouts, modify the extraction logic in pymupdf_rag.py or the preprocessing steps in pdf.py before the LLM calls begin.

Editing Section-Wise Prompts

The TemplateManager class in prompts/template_manager.py (lines 35-48) dynamically loads and renders Jinja templates for each resume section. The PDFHandler calls TemplateManager.render_template to generate prompts for extracting basics, work history, education, skills, projects, and awards.

To customize what information the LLM extracts, edit the corresponding Jinja file in prompts/templates/:

# prompts/templates/work.jinja (excerpt)

{{ text_content }}

# New instruction:

Please also extract any mention of **remote-work experience** and include it under a new key "remote_experience".

The TemplateManager automatically picks up template changes on the next run, allowing rapid iteration without restarting services.

Enhancing Candidate Enrichment

GitHub Profile Integration

The github.py module handles fetching, filtering, and classifying GitHub profiles and repositories. It parses usernames from resumes, calls the GitHub API, and uses the github_project_selection.jinja template to select the top 7 projects for evaluation. These results merge into the JSONResume object before scoring.

To adjust how the system evaluates open-source contributions—such as filtering by star count, repository age, or language—you can modify the API logic in github.py or the project selection criteria in the Jinja template.

Adjusting Evaluation Criteria and Scoring

Scoring Weights and Constants

The evaluator.py file contains the core scoring logic and fairness rules. Key constants like MAX_BONUS_POINTS (default 20) control the scoring boundaries. You can adjust these values to change how aggressively the system weights certain achievements:

# evaluator.py (excerpt)

MAX_BONUS_POINTS = 30      # increased from 20

# Later in the scoring logic:

if evaluation_data.open_source > 5:
    evaluation_data.final_score += 5  # extra bonus for strong OSS record

Evaluation Prompts

The evaluator builds prompts using resume_evaluation_criteria.jinja and resume_evaluation_system_message.jinja (referenced in prompts/template_manager.py lines 46-48 and evaluator.py lines 48-60). Modifying these templates changes how the LLM judges candidate quality, fairness criteria, and category weights. The parsed response converts into an EvaluationData object, so ensure your template changes align with the expected JSON schema.

Configuring LLM Providers and Runtime Behavior

Switching Between Ollama and Gemini

Provider configuration lives in config.py and models.py, where MODEL_PROVIDER_MAPPING defines available backends. To switch from the default Ollama to Gemini, set environment variables:

# .env (or copy from .env.example)

LLM_PROVIDER=gemini
DEFAULT_MODEL=gemini-2.5-pro
GEMINI_API_KEY=YOUR_KEY_HERE

Changing the provider in config.py affects how the LLM is invoked but does not impact the rest of the workflow logic, making it safe to experiment with different models.

CLI and Output Customization

The score.py file serves as the main entry point and orchestration layer. It wires all stages together and handles the CSV export logic. You can modify score.py to insert additional processing steps, replace the CSV writer with a database sink, or add new command-line arguments. The DEVELOPMENT_MODE flag in config.py enables debug output and verbose logging for testing customizations.

Summary

• Template customization: Edit Jinja files in prompts/templates/ to change extraction or evaluation prompts; changes reload automatically via TemplateManager.
• Scoring adjustments: Modify MAX_BONUS_POINTS and scoring logic in evaluator.py to adjust fairness rules and category weights.
• Data extraction: Update pymupdf_rag.py and pdf.py to change how raw PDFs convert to structured text.
• GitHub enrichment: Customize github.py and github_project_selection.jinja to filter repositories differently.
• Provider switching: Set LLM_PROVIDER and DEFAULT_MODEL in config.py or environment variables to switch between Ollama and Gemini.
• Orchestration: Extend score.py to add new output formats or processing steps in the pipeline.

Frequently Asked Questions

How do I add a new evaluation category to the scoring system?

Create a new Jinja template in prompts/templates/ for the category, then update evaluator.py to parse the new field from the LLM response and apply scoring weights. Ensure the template outputs valid JSON that matches the EvaluationData structure expected by the scorer.

Can I use a different LLM provider without changing the evaluation logic?

Yes. Modify config.py and the MODEL_PROVIDER_MAPPING in models.py to add your provider credentials. The provider configuration is decoupled from the evaluation logic, so switching between Ollama, Gemini, or other providers only requires changing environment variables in your .env file.

Where does the pipeline handle the GitHub repository selection?

The github.py module fetches repository data and uses the github_project_selection.jinja template to rank and select the top 7 projects. These merge into the candidate profile before evaluator.py processes the final score.

How do I enable debug mode during customization?

Set DEVELOPMENT_MODE to True in config.py. This flag increases logging verbosity and enables additional output that helps verify your template changes and scoring adjustments are working correctly during development.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}