How Are Tests Run in the Hiring-Agent Repository? A Guide to Smoke Checks and Manual Validation

The hiring-agent repository does not use pytest or unittest; instead, it relies on manual smoke-check scripts that exercise each processing stage by running the CLI against sample PDFs.

The interviewstreet/hiring-agent codebase handles the end-to-end pipeline of parsing resumes, enriching them with GitHub data, and evaluating candidates. Because the project lacks a conventional automated test suite, understanding how are tests run in the hiring-agent repository requires familiarity with its smoke-check workflow and manual verification steps.

Testing Approach: Smoke Checks vs. Automated Suites

The repository deliberately avoids traditional unit test frameworks. Instead, validation occurs through executable scripts that process real data through each pipeline stage.

Why There Is No Traditional Test Suite

According to the source code analysis, the project ships no formal test files like test_*.py or *_test.py. The complexity of LLM-dependent transformations—such as converting PDF sections to structured JSON—makes deterministic unit testing difficult. Consequently, the repository treats manual execution of the processing pipeline as the primary verification method.

The Smoke-Check Methodology

Smoke checks in this context involve running the actual entry points with minimal inputs to verify that components execute without errors. The repository breaks down into discrete stages, each testable via specific scripts:

• PDF → Markdown: Run pymupdf_rag.py or pdf.py against a small PDF file
• Section → JSON Resume: Execute pdf.py to invoke LLM prompting per section and build a JSONResume object
• GitHub Enrichment: Run github.py with a known username to fetch profile data
• Evaluation: Invoke evaluator.py on enriched JSON to obtain scores
• End-to-End: Use score.py to orchestrate the complete pipeline

How to Run the Smoke Checks

Running the smoke-check suite requires three sequential steps: dependency installation, environment configuration, and manual execution of the pipeline or individual components.

Prerequisites and Environment Setup

First, install the required dependencies:

pip install -r requirements.txt

Next, configure the environment variables required for LLM providers and GitHub API access:

cp .env.example .env

# Edit .env to set LLM_PROVIDER, DEFAULT_MODEL, GEMINI_API_KEY, and GITHUB_TOKEN

Executing the End-to-End Pipeline

The primary smoke test runs the CLI entry point score.py against a sample resume:

python score.py examples/sample_resume.pdf

This command exercises the full stack: PDF parsing, LLM-based section extraction, GitHub enrichment, and final evaluation. The script outputs a human-readable evaluation to stdout, confirming that all integrated components function correctly.

Testing Individual Components

For targeted debugging, run individual stage scripts:

• PDF conversion: python pymupdf_rag.py sample.pdf
• GitHub fetch: python github.py <username>
• Evaluation logic: python evaluator.py <enriched_json_path>

These granular checks isolate failures to specific modules without executing the full pipeline.

Key Files Involved in Testing

Understanding the testing workflow requires familiarity with these specific source files:

• score.py: The CLI entry point that orchestrates the complete pipeline and serves as the primary smoke-test interface
• pymupdf_rag.py: Handles low-level PDF-to-markdown conversion using PyMuPDF
• pdf.py: Manages PDF reading, LLM prompting per section, and JSONResume assembly via Pydantic models
• github.py: Fetches and classifies GitHub repository data for resume enrichment
• evaluator.py: Implements fairness-aware scoring logic on the enriched candidate data
• models.py: Contains Pydantic schemas and provider-agnostic LLM wrappers used across the pipeline
• CONTRIBUTING.md: Guidelines for adding new smoke-check scripts and verification steps

Development Mode and Caching

When DEVELOPMENT_MODE=True (the default setting), the pipeline caches intermediate JSON artifacts under the cache/ directory and appends evaluation results to resume_evaluations.csv. This behavior facilitates iterative smoke testing by preserving LLM outputs and allowing inspection of transformation stages without re-running expensive API calls.

Contributing New Smoke Checks

The repository encourages contributors to add smoke-check scripts that call pipeline stages with minimal inputs. As documented in CONTRIBUTING.md, pull requests should include verification steps demonstrating that each modified component executes successfully against representative test data. Adding a formal pytest suite remains a documented future contribution opportunity.

Summary

• The hiring-agent repository uses smoke-check scripts rather than pytest or unittest for verification
• Run the complete pipeline with python score.py <path_to_pdf> to test end-to-end functionality
• Test individual components by executing pymupdf_rag.py, pdf.py, github.py, or evaluator.py directly
• Set DEVELOPMENT_MODE=True to enable caching of intermediate JSON results under cache/
• No formal automated test suite exists; contributors verify changes through manual CLI execution

Frequently Asked Questions

Does the hiring-agent repository have unit tests?

No, the repository does not contain unit tests or automated test suites. Verification relies entirely on manual smoke-check scripts that execute the actual processing pipeline against sample PDFs and known data inputs.

How do I verify that my changes work correctly?

Run the CLI entry point python score.py examples/sample_resume.pdf to execute the full pipeline. For targeted testing, run individual stage scripts like python github.py <username> or python evaluator.py <json_path> to isolate specific components.

What is the purpose of the DEVELOPMENT_MODE environment variable?

When DEVELOPMENT_MODE=True (the default), the pipeline caches intermediate JSON outputs to the cache/ directory and logs evaluations to resume_evaluations.csv. This facilitates iterative smoke testing by avoiding redundant LLM API calls and preserving transformation artifacts for inspection.

Where can I find guidelines for adding tests?

Refer to CONTRIBUTING.md in the repository root. This file outlines expectations for adding new smoke-check scripts and verification steps, though it notes that implementing a formal pytest-based test suite is currently left as a future contribution opportunity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}