How Tests Are Structured and Run in the AI Job Search Project

TLDR: The AI Job Search repository uses a conventional Python unittest framework organized under a top-level tests/ directory, complemented by GitHub Actions CI that also validates TypeScript CLI tools and LaTeX compilation.

The MadsLorentzen/ai-job-search project maintains code quality through a comprehensive testing strategy that combines Python's standard library with modern JavaScript tooling. Understanding how tests are structured and run enables contributors to validate changes locally before submitting pull requests. The suite covers everything from PDF verification logic in tools/verify_pdf.py to CLI type safety and LaTeX compilation pipelines.

Test Organization and Structure

Python Unit Test Layout

All Python test modules reside under the top-level tests/ directory, following the standard test_*.py naming convention required by unittest discovery. The project uses the standard library's unittest framework exclusively, avoiding external test dependencies.

Tests typically follow a three-phase pattern:

1. Set up temporary resources using tempfile.TemporaryDirectory for isolated filesystem operations.
2. Patch external tool execution (e.g., tools.verify_pdf.run_tool) with unittest.mock.patch to isolate logic from external dependencies.
3. Assert expected behavior or verify that VerificationError is raised for error paths.

Key Test Modules and Coverage

The test suite focuses on critical tooling and command logic:

• test_verify_pdf.py: Validates the PDF-verification helper implemented in tools/verify_pdf.py, ensuring character extraction and minimum content thresholds work correctly.
• test_security_guards.py: Ensures tools/security_guards.py enforces permission rules, .gitignore policies, and manifest validation for CI security.
• test_salary_lookup.py: Exercises the salary-benchmarking utility (salary_lookup.py) for accurate data retrieval.
• test_html_report_command.py: Checks the /html-report command's CSV-to-HTML conversion logic.

Additional modules cover CLI linting, outcome follow-up logic, and Notion synchronization components.

How to Run Tests Locally

Executing Python Unit Tests

Run the complete test suite using Python's built-in discovery mechanism from the repository root:

python -m unittest discover -s tests -t . -v

The flags specify:

• -s tests: Search directory for test modules.
• -t .: Project root as the top-level package for imports.
• -v: Verbose output showing individual test names and results.

For targeted debugging, execute specific modules directly:

python -m unittest tests/test_verify_pdf.py

Testing TypeScript CLI Tools

The repository includes job-portal CLI tools under .agents/skills/*/cli/ directories. To validate these locally as the CI does:

cd .agents/skills/jobindex-search/cli
bun install
bun run typecheck
bun test

Replicating LaTeX Smoke Tests Locally

The CI validates PDF generation by compiling LaTeX documents. To mirror this locally without installing texlive:

docker run --rm -v "$(pwd)":/repo -w /repo texlive/texlive:latest \
  /bin/bash -c "\
    cd cv && lualatex -interaction=nonstopmode -halt-on-error main_example.tex && \
    cd ../cover_letters && xelatex -interaction=nonstopmode -halt-on-error cover_example.tex && \
    python3 tools/verify_pdf.py cv/main_example.pdf --min-chars 100 && \
    python3 tools/verify_pdf.py cover_letters/cover_example.pdf --min-chars 100"

CI/CD Test Execution Pipeline

GitHub Actions Workflow Structure

The .github/workflows/ci.yml orchestrates validation across four distinct jobs:

1. lint: Executes python tools/lint_skills.py to validate skill and command definitions against project standards.
2. security-guards: Runs python tools/security_guards.py to verify file permissions and repository policies.
3. python-tests: Invokes python -m unittest discover -s tests -t . -v for comprehensive Python test coverage with verbose reporting.
4. cli-checks: Iterates through each job-portal CLI directory (freehire-search, jobbank-search, etc.), installs dependencies with Bun, executes TypeScript type-checking (bun run typecheck), and runs fixture tests (bun test).

Production-Like PDF Verification

The CI includes a LaTeX smoke test that compiles example CV and cover-letter PDFs using lualatex and xelatex, then validates them with tools/verify_pdf.py. This demonstrates how the same verification logic exercised in unit tests also functions in a production-like pipeline environment.

Summary

• Tests are organized under tests/ using the test_*.py naming convention with Python's unittest framework.
• Run locally with python -m unittest discover -s tests -t . or target specific files like tests/test_verify_pdf.py.
• The CI pipeline in .github/workflows/ci.yml validates Python logic, security policies (tools/security_guards.py), and TypeScript CLI type safety.
• CLI tools require Bun for dependency management (bun install), type checking (bun run typecheck), and test execution (bun test).
• LaTeX smoke tests ensure PDF generation integrity alongside Python unit tests.

Frequently Asked Questions

What testing framework does the AI Job Search project use?

The project uses Python's standard library unittest framework exclusively. Tests typically employ tempfile.TemporaryDirectory for setup and unittest.mock.patch to isolate external tool calls like tools.verify_pdf.run_tool. Assertions verify expected behavior or catch VerificationError exceptions for error paths.

How do I run only a specific test file instead of the full suite?

Specify the file path directly to the unittest module: python -m unittest tests/test_verify_pdf.py. This executes only the verification logic in that module without running the discovery process across the entire tests/ directory.

Does the CI validate the TypeScript CLI tools?

Yes. The GitHub Actions workflow includes a cli-checks job that iterates through each skill's CLI directory under .agents/skills/*/cli/, runs bun install, executes bun run typecheck for static analysis, and calls bun test for any bundled fixture tests.

How are PDF generation and verification tested?

The CI runs a LaTeX smoke test that compiles example documents using lualatex and xelatex, then validates the output with python3 tools/verify_pdf.py --min-chars 100. This ensures both the LaTeX templates and the PDF verification logic function correctly in a containerized environment matching production conditions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}