# How to Run Tests for the i-have-adhd Project: A Complete Guide

> Learn how to run tests for the i-have-adhd project. This guide provides simple steps to execute all test cases and validate the evaluation harness efficiently.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: how-to-guide
- Published: 2026-07-30

---

**Run `python -m unittest discover -s tests` from the repository root to execute the 9 test cases in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) that validate the evaluation harness.**

The `ayghri/i-have-adhd` repository ships with a built-in test suite that validates the evaluation harness implemented in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py). These tests use Python’s standard `unittest` framework, requiring no third-party dependencies or complex configuration. Running the test suite ensures that case loading, score aggregation, and release gate logic function correctly before deploying the evaluation pipeline.

## Prerequisites

Before executing tests, ensure you have **Python 3.9 or newer** installed, as the codebase utilizes modern type hints incompatible with earlier versions. Clone the repository and navigate to the root directory containing the `scripts/` and `tests/` subdirectories.

## Running the Full Test Suite

To execute all tests at once, use Python’s unittest discovery mechanism from the repository root:

```bash
python -m unittest discover -s tests

```

This command automatically locates and runs all `test_*.py` files in the `tests/` directory. Successful execution displays confirmation that 9 tests passed:

```

.........
----------------------------------------------------------------------
Ran 9 tests in 0.12s

OK

```

For verbose output showing individual test names and docstrings, append the `-v` flag:

```bash
python -m unittest -v discover -s tests

```

## Running Individual Test Files

To focus on specific functionality without running the entire suite, target a single test file directly:

```bash
python -m unittest tests/test_run_evals.py

```

This executes only the evaluation harness tests, providing faster feedback when debugging specific validation logic.

## What the Tests Validate

The test suite in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) exercises nine critical validation scenarios covering the core utilities in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py):

- **`test_case_catalog_is_valid_and_balanced`** – Loads `evals/cases.jsonl` and verifies the file is well-formed JSONL containing a diverse set of evaluation cases.

- **`test_score_summary_applies_weights_and_release_gates`** – Calls `run_evals.summarize_scores` with minimal score rows to validate weighted score calculations and release-gate logic.

- **`test_candidate_blocker_fails_release_gate`** – Ensures that blocker flags in candidate rows correctly force the release gate to fail.

- **`test_conditions_judged_on_different_cases_are_rejected`** – Verifies that the summarizer raises an error when baseline and candidate rows reference different evaluation cases.

- **`test_duplicate_score_rows_are_rejected`** – Confirms that duplicate rows in score data raise a `ValueError`.

- **`test_duplicate_case_ids_are_rejected`** – Validates that the case validator catches duplicate `id` fields within the case catalog.

- **`test_jsonl_loader_reports_invalid_rows`** – Confirms that `run_evals.read_jsonl` raises clear errors when encountering malformed JSON lines.

- **`test_unmetered_runner_is_rejected_before_any_call`** – Ensures the runner aborts early when unmetered calls are not allowed, preventing unnecessary API invocations.

- **`test_completed_keys_support_resuming_partial_runs`** – Tests the helper function that tracks which evaluation rows have already been written, enabling resumable evaluation runs.

## Key Files and Functions Tested

The tests validate the evaluation pipeline in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py), which implements:

- **Case management** – Loading and validating entries from `evals/cases.jsonl`
- **Prompt building** – Constructing evaluation prompts for LLM runners
- **Score aggregation** – Weighted scoring calculations via `summarize_scores`
- **Data parsing** – Robust JSONL reading through `read_jsonl`
- **CLI commands** – Indirect testing of the `validate`, `plan`, `score`, and `run` subcommands

## Summary

- The `i-have-adhd` project uses Python’s built-in `unittest` framework with no external test runners required.
- Run `python -m unittest discover -s tests` from the repository root to execute all 9 tests.
- Individual test files can be targeted with `python -m unittest tests/test_run_evals.py`.
- Tests validate [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) functionality including case loading, scoring logic, and error handling.
- All tests complete in approximately 0.12 seconds, providing rapid feedback during development.

## Frequently Asked Questions

### Do I need to install pytest or other testing frameworks to run these tests?

No. The repository uses Python’s standard library `unittest` module exclusively. You can execute all tests using the built-in test runner without installing third-party packages like pytest, nose, or unittest2.

### What Python version is required to run the test suite?

Python 3.9 or newer is required. The codebase utilizes type hints and syntax features introduced in recent Python versions, and running the tests on older versions will likely result in syntax errors or import failures when loading [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py).

### Why do I need to run the tests from the repository root?

The tests reference files using relative paths (such as `evals/cases.jsonl` and [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py)). Running from the root directory ensures these path references resolve correctly when the test loader imports modules and the test cases access data files on disk.

### Can I test the CLI commands directly instead of using unittest?

While the repository provides a `run_evals` CLI with `validate`, `plan`, `score`, and `run` subcommands, the recommended approach is running the unittest suite. The tests indirectly exercise these CLI functions through their exposed Python APIs, providing faster and more isolated feedback than full CLI integration testing.