How to Run Unit Tests, Evals, and Plugin Validation in the i‑have‑adhd Repository

Execute python3 -m unittest discover -s tests -v to run unit tests, python3 scripts/run_evals.py validate to check evaluation cases, and claude plugin validate . to confirm the plugin loads correctly in Claude Code.

The ayghri/i-have-adhd repository provides a Claude Code plugin designed for ADHD‑focused developer workflows. Before submitting changes, contributors must run three distinct verification commands to ensure code correctness, evaluation integrity, and runtime compatibility.

Running the Unit Test Suite

The primary method for executing Python‑based regressions is the built‑in unittest discovery command.

python3 -m unittest discover -s tests -v

This command recursively executes all tests under the tests/ directory, including tests/test_always_on_hooks.py, tests/test_run_evals.py, and tests/test_opencode_plugin.py. According to the source code in CONTRIBUTING.md and AGENTS.md, this suite validates core library functions, hook implementations in hooks/, and skill markdown syntax.

The repository’s CI workflow (.github/workflows/plugin-load-check.yml) specifically invokes python -m unittest tests.test_always_on_hooks -v to verify that always‑on hooks load without error.

Validating Evaluations with run_evals.py

To ensure evaluation cases are well‑formed and scoring logic processes them correctly, run the lightweight validator implemented in scripts/run_evals.py.

python3 scripts/run_evals.py validate

This command parses evals/cases.jsonl and any associated JSONL result files, checking for required fields, valid value ranges, and proper pairing logic between baseline and candidate conditions. As implemented in the repository, the validate sub‑command prevents runtime crashes during future run or score operations by catching malformed data early.

Documentation for this step appears in CONTRIBUTING.md, AGENTS.md, and the evals/README.md file.

Verifying Plugin Installation with Claude Code

Plugin validation confirms that the manifest at .claude-plugin/plugin.json is syntactically correct and that the runtime can reach an "enabled" state.

claude plugin validate .

This command requests Claude Code’s plugin manager to parse the manifest, install the plugin from the current checkout, and verify initialization. It catches missing files, incorrect front‑matter, or schema mismatches before deployment. Note that this step requires a local Claude Code installation (npm install -g @anthropic-ai/claude-code).

The command is documented in AGENTS.md as the final verification step before submitting pull requests.

Summary

• Unit tests: Run python3 -m unittest discover -s tests -v to check core logic and hooks.
• Evaluation validation: Run python3 scripts/run_evals.py validate to verify evals/cases.jsonl structure.
• Plugin validation: Run claude plugin validate . to ensure the Claude Code plugin loads correctly.

Frequently Asked Questions

What is the exact command to run all unit tests in i‑have‑adhd?

Use python3 -m unittest discover -s tests -v. This discovers and executes all test modules under the tests/ directory, including specific coverage for always‑on hooks and OpenCode integration.

How do I validate evaluation cases before running a full evaluation?

Execute python3 scripts/run_evals.py validate. This script checks that evals/cases.jsonl contains properly formatted JSON lines with all required fields and valid value ranges without executing the actual evaluation logic.

Why does plugin validation require Claude Code to be installed?

The command claude plugin validate . invokes the Claude Code CLI to parse .claude-plugin/plugin.json, perform a test installation, and confirm the plugin reaches an "enabled" state. Without the CLI, the runtime cannot verify manifest compatibility or load‑time errors.

Can I run unit tests and eval validation on Windows or macOS?

Yes. The first two commands (unittest and run_evals.py) run in any Python 3 environment across platforms. Only the claude plugin validate . command requires the Claude Code CLI, which is platform‑agnostic but must be installed separately via npm.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}