InterviewStreet Hiring Agent Repository: Architecture, Workflow, and Key Components

The InterviewStreet Hiring Agent is a self‑contained Python pipeline that turns a résumé PDF into a structured evaluation score using LLM‑powered parsing, GitHub enrichment, and fairness‑aware scoring.

The interviewstreet/hiring-agent repository implements a linear data‑flow architecture designed for reproducible, explainable résumé evaluation. It extracts content from PDFs, validates it against the JSON‑Resume schema, enriches it with GitHub metadata, and produces a typed EvaluationData object with granular scoring categories. The system supports both local (Ollama) and cloud (Google Gemini) LLM backends via a configurable provider abstraction.

Five‑Stage Pipeline Architecture

The Hiring Agent orchestrates five distinct stages, each isolated in dedicated modules to ensure clear separation of concerns.

1. PDF Extraction and Text Conversion

The pipeline begins in pymupdf_rag.py and pdf.py, where the PDFHandler class uses PyMuPDF to read each page of a PDF and convert it to Markdown‑like text. The handler then calls the LLM for every résumé section (Basics, Work, Education, Skills, Projects, Awards).

• Source files: [pymupdf_rag.py](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py), [pdf.py](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py)

2. Section Parsing with Jinja Templates

Strictly‑typed prompts reside in prompts/templates/*.jinja. Each template encodes a specific section (Basics, Work, Education, Skills, Projects, Awards) and instructs the LLM to return JSON conforming to the JSON‑Resume schema.

• Source: prompts/templates/basics.jinja (and similar section files)

3. GitHub Repository Enrichment

The github.py module detects a GitHub profile URL in the Basics section, fetches the user profile and public repositories via the GitHub API, and asks the LLM to select the seven most relevant projects for evaluation.

• Source: [github.py](https://github.com/interviewstreet/hiring-agent/blob/main/github.py)

4. Fairness‑Aware Evaluation

evaluator.py invokes the LLM with fairness‑aware prompts to generate an EvaluationData object. Scores split into four categories: open‑source, self‑projects, production, and technical‑skills, plus optional bonus points and deductions.

• Source: [evaluator.py](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)

5. Orchestration and Output

score.py coordinates the entire pipeline, caches intermediate JSON when DEVELOPMENT_MODE is enabled, prints a human‑readable report, and writes a CSV row to resume_evaluations.csv for downstream analytics.

• Source: [score.py](https://github.com/interviewstreet/hiring-agent/blob/main/score.py)

Core Data Models in models.py

All data structures are strictly defined in models.py using Pydantic, providing runtime validation and type safety across the pipeline.

JSONResume Schema

The JSONResume class represents the top‑level parsed résumé, comprising optional sub‑models such as Basics, Work, Education, and Projects.

• Source: [models.py – JSONResume](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L202)

EvaluationData

The EvaluationData class encapsulates the final evaluation, containing nested Scores, BonusPoints, Deductions, and lists of strengths versus improvement areas.

• Source: [models.py – EvaluationData](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L44)

LLM Provider Abstractions

The repository defines an LLMProvider protocol with two concrete implementations:

• OllamaProvider – for local inference (default)
• GeminiProvider – for Google Gemini API access

Provider selection is driven by the LLM_PROVIDER environment variable.

• Sources: [models.py – OllamaProvider](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L71), [models.py – GeminiProvider](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L13)

Configuration and Environment Variables

Runtime behavior is controlled via a .env.example file and config.py:

• LLM_PROVIDER – set to ollama (default) or gemini

• DEFAULT_MODEL – model name passed to the provider (e.g., gemma3:4b or gemini-2.5-pro)

• GEMINI_API_KEY – required only when LLM_PROVIDER=gemini

• DEVELOPMENT_MODE – flag in config.py that enables caching of intermediate results and automatic CSV export

• Source: [config.py](https://github.com/interviewstreet/hiring-agent/blob/main/config.py#L1)

Step‑by‑Step Execution Flow

The score.py module implements a strict seven‑step workflow:

1. Cache check – If DEVELOPMENT_MODE is true and a cache file exists, the résumé JSON is loaded (score.py lines 26‑35).
2. PDF → JSONResume – PDFHandler.extract_json_from_pdf parses the PDF and validates core sections (score.py lines 51‑58).
3. GitHub data – fetch_and_display_github_info executes only when a GitHub profile is present (score.py lines 97‑114).
4. Resume text assembly – The JSON résumé, optional GitHub data, and optional blog data are concatenated into a single prompt (score.py lines 70‑82).
5. Evaluation – ResumeEvaluator.evaluate_resume sends the assembled text to the LLM and returns a typed EvaluationData (score.py line 84).
6. Reporting – print_evaluation_results formats the scores, bonuses, deductions, strengths, and improvement areas for console output (score.py lines 30‑60).
7. CSV export – When in development mode, the transformed evaluation row is appended to resume_evaluations.csv (score.py lines 51‑63).

Extending the Pipeline

The Hiring Agent architecture supports two primary extension vectors:

• Adding a new LLM provider – Implement a class that follows the LLMProvider protocol, expose it in prompt.py or models.py, and add the mapping to MODEL_PARAMETERS.
• New résumé sections – Create a Jinja template under prompts/templates/, extend JSONResume with the corresponding field, and update pdf.py to call the LLM for the new section.

Running the Hiring Agent Pipeline

Execute the end‑to‑end evaluation with the following commands:

# 1. Install dependencies (Python 3.11+)

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 2. Configure the LLM backend

export LLM_PROVIDER=ollama               # or "gemini"

export DEFAULT_MODEL=gemma3:4b           # Ollama example

# For Gemini: export GEMINI_API_KEY=<your-key>

# 3. Run the pipeline on a résumé PDF

python score.py path/to/resume.pdf

Sample console output (truncated):

========================================================
📊 RESUME EVALUATION RESULTS FOR: Jane Doe
========================================================

🎯 OVERALL SCORE: 87.5/110

📈 DETAILED SCORES:
--------------------------------------------
🌐 Open Source:          30/35
   Evidence: Contributed to 3 OSS libs

🚀 Self Projects:        22/30
   Evidence: Built a micro‑service

🏢 Production Experience: 25/25
   Evidence: 2 years at Acme Corp

💻 Technical Skills:     8/10
   Evidence: Proficient in Python, Docker

⭐ BONUS POINTS: 5.0
   - Leadership role, open‑source mentorship
...

Summary

• The InterviewStreet Hiring Agent is a Python 3.11+ pipeline using PyMuPDF, Pydantic, and LLM providers to evaluate résumés.
• Architecture follows a five‑stage linear flow: PDF extraction → Section parsing → GitHub enrichment → Evaluation → Orchestration.
• Key entry points include score.py (CLI), pdf.py (parsing), github.py (enrichment), and evaluator.py (scoring).
• Data integrity is enforced through Pydantic models (JSONResume, EvaluationData) and Jinja‑templated prompts.
• Support for Ollama (local) and Gemini (cloud) backends is controlled via environment variables in config.py.

Frequently Asked Questions

How does the Hiring Agent handle GitHub repository validation?

The github.py module detects a GitHub profile URL in the Basics section of the parsed résumé, fetches the user profile and public repositories via the GitHub API, and uses the LLM to select the seven most relevant projects. This data is then fed into the evaluation context to verify claimed contributions against actual repository activity.

What is the difference between OllamaProvider and GeminiProvider?

OllamaProvider (defined in [models.py](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L71)) interfaces with local Ollama instances for offline inference, while GeminiProvider (defined in [models.py](https://github.com/interviewstreet/hiring-agent/blob/main/models.py#L13)) connects to Google's Gemini API. Both implement the LLMProvider protocol, allowing seamless switching via the LLM_PROVIDER environment variable without changing application code.

How do I add a new scoring category to the evaluation?

Extend the EvaluationData model in models.py to include the new category, create a corresponding Jinja template in prompts/templates/ to guide the LLM's output format, and update evaluator.py to aggregate the new scores into the final EvaluationData object returned by ResumeEvaluator.evaluate_resume.

Can I run the pipeline without an internet connection?

Yes, provided you set LLM_PROVIDER=ollama and have a local Ollama server running with the specified DEFAULT_MODEL (e.g., gemma3:4b). The GitHub enrichment step requires internet access; however, the pipeline will skip fetch_and_display_github_info if no GitHub profile is present in the résumé, allowing offline evaluation of local PDFs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}