# How Are Tests Run in the Hiring-Agent Repository? A Guide to Smoke Checks and Manual Validation > Discover how tests run in the hiring-agent repository. Learn about manual smoke-check scripts and CLI execution against sample PDFs for efficient validation. - Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent) - Tags: how-to-guide - Published: 2026-07-12 --- **The hiring-agent repository does not use pytest or unittest; instead, it relies on manual smoke-check scripts that exercise each processing stage by running the CLI against sample PDFs.** The interviewstreet/hiring-agent codebase handles the end-to-end pipeline of parsing resumes, enriching them with GitHub data, and evaluating candidates. Because the project lacks a conventional automated test suite, understanding how are tests run in the hiring-agent repository requires familiarity with its smoke-check workflow and manual verification steps. ## Testing Approach: Smoke Checks vs. Automated Suites The repository deliberately avoids traditional unit test frameworks. Instead, validation occurs through executable scripts that process real data through each pipeline stage. ### Why There Is No Traditional Test Suite According to the source code analysis, the project ships **no formal test files** like `test_*.py` or `*_test.py`. The complexity of LLM-dependent transformations—such as converting PDF sections to structured JSON—makes deterministic unit testing difficult. Consequently, the repository treats manual execution of the processing pipeline as the primary verification method. ### The Smoke-Check Methodology Smoke checks in this context involve running the actual entry points with minimal inputs to verify that components execute without errors. The repository breaks down into discrete stages, each testable via specific scripts: - **PDF → Markdown**: Run [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py) or [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) against a small PDF file - **Section → JSON Resume**: Execute [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) to invoke LLM prompting per section and build a `JSONResume` object - **GitHub Enrichment**: Run [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) with a known username to fetch profile data - **Evaluation**: Invoke [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) on enriched JSON to obtain scores - **End-to-End**: Use [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) to orchestrate the complete pipeline ## How to Run the Smoke Checks Running the smoke-check suite requires three sequential steps: dependency installation, environment configuration, and manual execution of the pipeline or individual components. ### Prerequisites and Environment Setup First, install the required dependencies: ```bash pip install -r requirements.txt ``` Next, configure the environment variables required for LLM providers and GitHub API access: ```bash cp .env.example .env # Edit .env to set LLM_PROVIDER, DEFAULT_MODEL, GEMINI_API_KEY, and GITHUB_TOKEN ``` ### Executing the End-to-End Pipeline The primary smoke test runs the CLI entry point [`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py) against a sample resume: ```bash python score.py examples/sample_resume.pdf ``` This command exercises the full stack: PDF parsing, LLM-based section extraction, GitHub enrichment, and final evaluation. The script outputs a human-readable evaluation to **stdout**, confirming that all integrated components function correctly. ### Testing Individual Components For targeted debugging, run individual stage scripts: - **PDF conversion**: `python pymupdf_rag.py sample.pdf` - **GitHub fetch**: `python github.py ` - **Evaluation logic**: `python evaluator.py ` These granular checks isolate failures to specific modules without executing the full pipeline. ## Key Files Involved in Testing Understanding the testing workflow requires familiarity with these specific source files: - **[`score.py`](https://github.com/interviewstreet/hiring-agent/blob/main/score.py)**: The CLI entry point that orchestrates the complete pipeline and serves as the primary smoke-test interface - **[`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py)**: Handles low-level PDF-to-markdown conversion using PyMuPDF - **[`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py)**: Manages PDF reading, LLM prompting per section, and `JSONResume` assembly via Pydantic models - **[`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py)**: Fetches and classifies GitHub repository data for resume enrichment - **[`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py)**: Implements fairness-aware scoring logic on the enriched candidate data - **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)**: Contains Pydantic schemas and provider-agnostic LLM wrappers used across the pipeline - **[`CONTRIBUTING.md`](https://github.com/interviewstreet/hiring-agent/blob/main/CONTRIBUTING.md)**: Guidelines for adding new smoke-check scripts and verification steps ## Development Mode and Caching When `DEVELOPMENT_MODE=True` (the default setting), the pipeline caches intermediate JSON artifacts under the `cache/` directory and appends evaluation results to `resume_evaluations.csv`. This behavior facilitates iterative smoke testing by preserving LLM outputs and allowing inspection of transformation stages without re-running expensive API calls. ## Contributing New Smoke Checks The repository encourages contributors to add smoke-check scripts that call pipeline stages with minimal inputs. As documented in [`CONTRIBUTING.md`](https://github.com/interviewstreet/hiring-agent/blob/main/CONTRIBUTING.md), pull requests should include verification steps demonstrating that each modified component executes successfully against representative test data. Adding a formal `pytest` suite remains a documented future contribution opportunity. ## Summary - The hiring-agent repository uses **smoke-check scripts** rather than pytest or unittest for verification - Run the complete pipeline with `python score.py ` to test end-to-end functionality - Test individual components by executing [`pymupdf_rag.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pymupdf_rag.py), [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py), [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py), or [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) directly - Set `DEVELOPMENT_MODE=True` to enable caching of intermediate JSON results under `cache/` - No formal automated test suite exists; contributors verify changes through manual CLI execution ## Frequently Asked Questions ### Does the hiring-agent repository have unit tests? No, the repository does not contain unit tests or automated test suites. Verification relies entirely on manual smoke-check scripts that execute the actual processing pipeline against sample PDFs and known data inputs. ### How do I verify that my changes work correctly? Run the CLI entry point `python score.py examples/sample_resume.pdf` to execute the full pipeline. For targeted testing, run individual stage scripts like `python github.py ` or `python evaluator.py ` to isolate specific components. ### What is the purpose of the DEVELOPMENT_MODE environment variable? When `DEVELOPMENT_MODE=True` (the default), the pipeline caches intermediate JSON outputs to the `cache/` directory and logs evaluations to `resume_evaluations.csv`. This facilitates iterative smoke testing by avoiding redundant LLM API calls and preserving transformation artifacts for inspection. ### Where can I find guidelines for adding tests? Refer to [`CONTRIBUTING.md`](https://github.com/interviewstreet/hiring-agent/blob/main/CONTRIBUTING.md) in the repository root. This file outlines expectations for adding new smoke-check scripts and verification steps, though it notes that implementing a formal pytest-based test suite is currently left as a future contribution opportunity.