# How to Contribute to Testing the `i-have-adhd` Skill: A Complete Guide

> Learn how to contribute to testing the i-have-adhd skill. Follow our guide to add evaluation cases, run tests, and submit your pull request to improve the skill.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: how-to-guide
- Published: 2026-08-03

---

**Yes, you can contribute to testing the `i-have-adhd` skill by adding evaluation cases to `evals/cases.jsonl`, creating score fixtures, running the local test suite, and submitting a pull request.**

The `i-have-adhd` skill is a response-style plug-in that reshapes LLM output to better serve readers with ADHD. According to the ayghri/i-have-adhd source code, the testing framework is explicitly designed for community contributions, with clear extension points in `evals/cases.jsonl` and comprehensive unit tests in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py).

## Understanding the Test Architecture Before You Contribute

The testing framework splits responsibilities across four key components. Understanding these will help you target your contributions effectively.

| Component | Role | Key Source |
|-----------|------|------------|
| **Skill definition** | YAML-style rules file read by the harness and injected into prompts when the *condition* is *candidate* | [`skills/i-have-adhd/SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/skills/i-have-adhd/SKILL.md) |
| **Evaluation cases** | JSON-Lines catalog of prompts, risk levels, and success criteria | `evals/cases.jsonl` |
| **Test harness** | Loads cases, builds prompts, invokes runners, parses responses, aggregates scores | [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) |
| **Unit tests** | Validates harness behavior: validation, pairing, scoring, budget logic | [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) |

### How the Harness Orchestrates Test Runs

The [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) harness operates in three sequential stages:

1. **Validate the case catalog** — `validate_cases()` (lines 59-79) enforces required fields (`id`, `category`, `prompt`, `risk`, `criteria`) and checks for duplicate IDs
2. **Run evaluations** — `run_evaluations()` (lines 9-100) iterates over cases, builds prompts via `_condition_prompt()` (lines 71-82) inserting skill text when condition is *candidate*, then calls the configured runner
3. **Score results** — `summarize_scores()` (lines 30-68) applies weighted metrics and release-gate rules to determine if the candidate passes

## Four Ways to Contribute to `i-have-adhd` Skill Testing

### 1. Add New Evaluation Cases

The simplest entry point: extend `evals/cases.jsonl` with additional test scenarios.

Each case must follow this schema:

```json
{"id":"simple-math","category":"direct-answer","prompt":"What is 7 × 8?","risk":"low","criteria":["Answers 56."]}

```

**Required fields:**
- `id` — unique identifier (enforced by validation)
- `category` — classification for grouping results
- `prompt` — the actual input sent to the LLM
- `risk` — typically `low`, `medium`, or `high`
- `criteria` — array of strings defining success conditions

To add a case programmatically:

```python
import json, pathlib

new_case = {
    "id": "simple-math",
    "category": "direct-answer",
    "prompt": "What is 7 × 8?",
    "risk": "low",
    "criteria": ["Answers 56."]
}
cases_path = pathlib.Path("evals/cases.jsonl")
with cases_path.open("a", encoding="utf-8") as f:
    f.write(json.dumps(new_case, ensure_ascii=False) + "\n")

```

The unit test `test_case_catalog_is_valid_and_balanced` automatically verifies your addition.

### 2. Create Score Fixtures for Direct Scoring Tests

To test scoring logic without running live LLM calls, create `.jsonl` files mimicking runner output.

Each row must contain fields validated by `_validate_score()` (lines 82-95 in [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py)). Use these fixtures to verify edge cases in the scoring pipeline.

### 3. Run and Verify Your Changes Locally

Execute the full test suite before submitting:

```bash
python -m unittest discover -s tests

```

This validates:
- Case catalog integrity
- Field pairing logic
- Duplicate ID detection
- Budget handling
- Score aggregation accuracy

### 4. Run the Evaluation Harness Manually

Test your cases with actual skill injection:

```bash
python scripts/run_evals.py run \
  --runner-config evals/runners.example.json \
  --runner stub \
  --condition candidate \
  --condition-skill skills/i-have-adhd/SKILL.md \
  --output evals/out.jsonl

```

Then verify scoring:

```bash
python scripts/run_evals.py score evals/out.jsonl

```

This outputs weighted scores and the release-gate pass/fail decision.

## Submitting Your `i-have-adhd` Testing Contribution

1. **Fork** the ayghri/i-have-adhd repository
2. **Branch** from `main` with a descriptive name (`add-math-evaluation-cases`)
3. **Commit** your changes to `evals/cases.jsonl` and any new fixtures
4. **Push** and open a pull request

The CI pipeline ([`.github/workflows/plugin-load-check.yml`](https://github.com/ayghri/i-have-adhd/blob/main/.github/workflows/plugin-load-check.yml)) automatically runs the same test suite, blocking merges that break existing behavior.

## Key Files for Contributors

| Path | Purpose |
|------|---------|
| [`skills/i-have-adhd/SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/skills/i-have-adhd/SKILL.md) | 10 output-style rules defining the skill's behavior |
| `evals/cases.jsonl` | Extend this file with new test cases |
| [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) | Core harness with `validate_cases()`, `run_evaluations()`, `summarize_scores()` |
| [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) | Unit tests ensuring harness reliability |
| [`evals/runners.example.json`](https://github.com/ayghri/i-have-adhd/blob/main/evals/runners.example.json) | Template runner configuration |

## Summary

- **Yes, contributions are explicitly supported** — the framework is designed for community extension
- **Add cases to `evals/cases.jsonl`** — validated automatically by `validate_cases()` in [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py)
- **Run `python -m unittest discover -s tests`** — local verification before PR submission
- **Use `--condition candidate`** — to test skill injection via `_condition_prompt()`
- **CI enforces quality gates** — matching local test behavior exactly

## Frequently Asked Questions

### What programming skills do I need to contribute to `i-have-adhd` skill testing?

Basic Python and JSON familiarity suffice. Adding evaluation cases requires only editing `evals/cases.jsonl`. For deeper contributions, understanding the harness functions in [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) (particularly `_condition_prompt()` and `_validate_score()`) helps, but the existing unit tests provide guardrails.

### How does the harness inject the ADHD skill into test prompts?

The `_condition_prompt()` function (lines 71-82) conditionally prepends skill content from [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) when the `--condition` flag is set to `candidate`. This lets you compare baseline LLM responses against skill-modified output using identical evaluation cases.

### Can I test my cases without calling a live LLM?

Yes. The `--runner stub` option uses a mock runner. You can also create score fixtures and run `python scripts/run_evals.py score` directly against saved outputs, bypassing LLM calls entirely while still validating scoring logic.

### What makes a good evaluation case for the `i-have-adhd` skill?

Focus on prompts where ADHD-optimized formatting genuinely matters: long explanations, multi-step instructions, or dense technical content. The `risk` field should reflect real-world impact, and `criteria` must be objectively verifiable (e.g., "Uses bullet points" not "Is easy to read").