# How Tests Are Structured and Run in the AI Job Search Project

> Discover how tests are structured and run in the AI Job Search project. Learn about the Python unittest framework and GitHub Actions CI integration for efficient testing.

- Repository: [Mads Lorentzen/ai-job-search](https://github.com/MadsLorentzen/ai-job-search)
- Tags: internals
- Published: 2026-07-25

---

**TLDR:** The AI Job Search repository uses a conventional Python **unittest** framework organized under a top-level `tests/` directory, complemented by GitHub Actions CI that also validates TypeScript CLI tools and LaTeX compilation.

The MadsLorentzen/ai-job-search project maintains code quality through a comprehensive testing strategy that combines Python's standard library with modern JavaScript tooling. Understanding how tests are structured and run enables contributors to validate changes locally before submitting pull requests. The suite covers everything from PDF verification logic in [`tools/verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/verify_pdf.py) to CLI type safety and LaTeX compilation pipelines.

## Test Organization and Structure

### Python Unit Test Layout

All Python test modules reside under the top-level **`tests/`** directory, following the standard **`test_*.py`** naming convention required by unittest discovery. The project uses the standard library's **`unittest`** framework exclusively, avoiding external test dependencies.

Tests typically follow a three-phase pattern:

1. **Set up** temporary resources using `tempfile.TemporaryDirectory` for isolated filesystem operations.
2. **Patch** external tool execution (e.g., `tools.verify_pdf.run_tool`) with `unittest.mock.patch` to isolate logic from external dependencies.
3. **Assert** expected behavior or verify that `VerificationError` is raised for error paths.

### Key Test Modules and Coverage

The test suite focuses on critical tooling and command logic:

- **[`test_verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/test_verify_pdf.py)**: Validates the PDF-verification helper implemented in [`tools/verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/verify_pdf.py), ensuring character extraction and minimum content thresholds work correctly.
- **[`test_security_guards.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/test_security_guards.py)**: Ensures [`tools/security_guards.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/security_guards.py) enforces permission rules, `.gitignore` policies, and manifest validation for CI security.
- **[`test_salary_lookup.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/test_salary_lookup.py)**: Exercises the salary-benchmarking utility ([`salary_lookup.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/salary_lookup.py)) for accurate data retrieval.
- **[`test_html_report_command.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/test_html_report_command.py)**: Checks the `/html-report` command's CSV-to-HTML conversion logic.

Additional modules cover CLI linting, outcome follow-up logic, and Notion synchronization components.

## How to Run Tests Locally

### Executing Python Unit Tests

Run the complete test suite using Python's built-in discovery mechanism from the repository root:

```bash
python -m unittest discover -s tests -t . -v

```

The flags specify:
- **`-s tests`**: Search directory for test modules.
- **`-t .`**: Project root as the top-level package for imports.
- **`-v`**: Verbose output showing individual test names and results.

For targeted debugging, execute specific modules directly:

```bash
python -m unittest tests/test_verify_pdf.py

```

### Testing TypeScript CLI Tools

The repository includes job-portal CLI tools under `.agents/skills/*/cli/` directories. To validate these locally as the CI does:

```bash
cd .agents/skills/jobindex-search/cli
bun install
bun run typecheck
bun test

```

### Replicating LaTeX Smoke Tests Locally

The CI validates PDF generation by compiling LaTeX documents. To mirror this locally without installing texlive:

```bash
docker run --rm -v "$(pwd)":/repo -w /repo texlive/texlive:latest \
  /bin/bash -c "\
    cd cv && lualatex -interaction=nonstopmode -halt-on-error main_example.tex && \
    cd ../cover_letters && xelatex -interaction=nonstopmode -halt-on-error cover_example.tex && \
    python3 tools/verify_pdf.py cv/main_example.pdf --min-chars 100 && \
    python3 tools/verify_pdf.py cover_letters/cover_example.pdf --min-chars 100"

```

## CI/CD Test Execution Pipeline

### GitHub Actions Workflow Structure

The [`.github/workflows/ci.yml`](https://github.com/MadsLorentzen/ai-job-search/blob/main/.github/workflows/ci.yml) orchestrates validation across four distinct jobs:

1. **`lint`**: Executes `python tools/lint_skills.py` to validate skill and command definitions against project standards.
2. **`security-guards`**: Runs `python tools/security_guards.py` to verify file permissions and repository policies.
3. **`python-tests`**: Invokes `python -m unittest discover -s tests -t . -v` for comprehensive Python test coverage with verbose reporting.
4. **`cli-checks`**: Iterates through each job-portal CLI directory (`freehire-search`, `jobbank-search`, etc.), installs dependencies with **Bun**, executes TypeScript type-checking (`bun run typecheck`), and runs fixture tests (`bun test`).

### Production-Like PDF Verification

The CI includes a LaTeX smoke test that compiles example CV and cover-letter PDFs using `lualatex` and `xelatex`, then validates them with [`tools/verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/verify_pdf.py). This demonstrates how the same verification logic exercised in unit tests also functions in a production-like pipeline environment.

## Summary

- Tests are organized under **`tests/`** using the **`test_*.py`** naming convention with Python's **unittest** framework.
- Run locally with `python -m unittest discover -s tests -t .` or target specific files like [`tests/test_verify_pdf.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tests/test_verify_pdf.py).
- The CI pipeline in **[`.github/workflows/ci.yml`](https://github.com/MadsLorentzen/ai-job-search/blob/main/.github/workflows/ci.yml)** validates Python logic, security policies ([`tools/security_guards.py`](https://github.com/MadsLorentzen/ai-job-search/blob/main/tools/security_guards.py)), and TypeScript CLI type safety.
- CLI tools require **Bun** for dependency management (`bun install`), type checking (`bun run typecheck`), and test execution (`bun test`).
- LaTeX smoke tests ensure PDF generation integrity alongside Python unit tests.

## Frequently Asked Questions

### What testing framework does the AI Job Search project use?

The project uses Python's standard library **unittest** framework exclusively. Tests typically employ `tempfile.TemporaryDirectory` for setup and `unittest.mock.patch` to isolate external tool calls like `tools.verify_pdf.run_tool`. Assertions verify expected behavior or catch `VerificationError` exceptions for error paths.

### How do I run only a specific test file instead of the full suite?

Specify the file path directly to the unittest module: `python -m unittest tests/test_verify_pdf.py`. This executes only the verification logic in that module without running the discovery process across the entire `tests/` directory.

### Does the CI validate the TypeScript CLI tools?

Yes. The GitHub Actions workflow includes a `cli-checks` job that iterates through each skill's CLI directory under `.agents/skills/*/cli/`, runs `bun install`, executes `bun run typecheck` for static analysis, and calls `bun test` for any bundled fixture tests.

### How are PDF generation and verification tested?

The CI runs a LaTeX smoke test that compiles example documents using `lualatex` and `xelatex`, then validates the output with `python3 tools/verify_pdf.py --min-chars 100`. This ensures both the LaTeX templates and the PDF verification logic function correctly in a containerized environment matching production conditions.