i-have-adhd Skill Evaluations Testing Framework: Python unittest Implementation

The i-have-adhd repository validates its skill-evaluation logic using Python's built-in unittest framework, with all test suites located in the tests/ directory and executed via standard library test discovery.

The ayghri/i-have-adhd project relies on Python's standard testing library to ensure its skill evaluation pipeline functions correctly. Rather than introducing external dependencies like pytest, the codebase leverages the built-in unittest module to validate evaluation harnesses and plugin integrations. This approach keeps the testing infrastructure lightweight while providing robust validation for the ADHD skill assessment logic.

Standard Library Testing Architecture

unittest.TestCase Foundation

All validation logic in the i-have-adhd testing framework inherits from unittest.TestCase. In tests/test_run_evals.py, test classes like EvaluationHarnessTest extend this base class to leverage assertion methods such as assertEqual() and assertGreaterEqual(). The tests/test_opencode_plugin.py file follows the same pattern for plugin-specific validations.

Zero External Dependencies

The repository deliberately avoids third-party test runners. As implemented in ayghri/i-have-adhd, the unittest framework provides sufficient functionality for the project's needs without requiring additional packages in requirements.txt or pyproject.toml.

Key Files in the Testing Framework

The i-have-adhd skill evaluations testing framework organizes validation code across several critical paths:

• tests/test_run_evals.py: Contains EvaluationHarnessTest class with methods like test_case_catalog_is_valid_and_balanced() that validate the evaluation case loader in scripts/run_evals.py
• tests/test_opencode_plugin.py: Implements plugin-specific tests using the same unittest patterns
• scripts/run_evals.py: The core evaluation module exercised by the test suite, containing functions like load_cases() and summarize_scores()
• tests/: Root directory containing all unittest-based modules

Running the Skill Evaluation Tests

Execute the full test suite using Python's built-in discovery mechanism:

# From the repository root

python -m unittest discover -s tests -v

This command scans the tests/ directory for all classes inheriting from unittest.TestCase and generates verbose output showing individual test results. The -s tests flag specifically targets the test folder, ensuring the discovery process ignores other Python files in the repository.

Writing Tests for Evaluation Logic

When extending the i-have-adhd testing framework, developers create subclasses of unittest.TestCase and implement methods prefixed with test_. The following example demonstrates validating the score summarization logic:

import unittest
from scripts.run_evals import summarize_scores

class ScoreSummaryTest(unittest.TestCase):
    def test_weighted_score_calculation(self):
        dummy_scores = [
            {
                "case_id": "example",
                "trial": 1,
                "condition": "baseline",
                "correctness": 4,
                "autonomy": 3,
                "actionability": 4,
                "safety": 5,
                "concision": 4,
                "blocker": False,
                "notes": "demo",
            },
            # ... additional rows for candidate condition ...

        ]
        summary = summarize_scores(dummy_scores)
        self.assertIn("weighted_score", summary["conditions"]["baseline"])

This pattern mirrors the existing implementation in tests/test_run_evals.py, where test_case_catalog_is_valid_and_balanced() calls run_evals.load_cases() and asserts that validation errors return an empty list while requiring at least twelve defined cases.

Summary

• The i-have-adhd repository uses Python's built-in unittest framework for all skill evaluation testing
• Test classes inherit from unittest.TestCase and reside in the tests/ directory
• No external test runners like pytest are required; standard library discovery handles execution
• Key test files include tests/test_run_evals.py and tests/test_opencode_plugin.py
• Tests validate core functions in scripts/run_evals.py including load_cases() and summarize_scores()

Frequently Asked Questions

What testing framework does i-have-adhd use for skill evaluations?

The project uses Python's built-in unittest framework exclusively. All test modules import unittest and define test cases as subclasses of unittest.TestCase, located in the tests/ directory. This standard library approach eliminates external testing dependencies while providing comprehensive validation for the skill evaluation pipeline.

Does i-have-adhd require pytest or other third-party test runners?

No. The repository relies entirely on the standard library's unittest module. Tests are discovered and executed using python -m unittest discover, which ships with every Python installation. This design choice keeps the project's dependency footprint minimal and avoids version conflicts with external testing tools.

How do I run the skill evaluation tests locally?

Navigate to the repository root and execute python -m unittest discover -s tests -v. The -s tests flag specifies the test directory, while -v enables verbose output showing individual test names and results. This command runs all test cases in test_run_evals.py and test_opencode_plugin.py without requiring additional configuration files.

Where are the main test files located in the i-have-adhd repository?

The primary test files are tests/test_run_evals.py (containing EvaluationHarnessTest and validation logic for the evaluation harness) and tests/test_opencode_plugin.py (testing the OpenCode plugin integration). Both files import from scripts/run_evals.py, which contains the core evaluation functions being tested.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}