# i-Have-ADHD Skill Behaviors Tested: Rules, Evaluation Harness, and Compliance Validation

> Discover how the i-have-adhd skill tests its rules using an evaluation harness and JSON test cases. Learn about compliance validation and verified behaviors.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: deep-dive
- Published: 2026-08-03

---

**The i-have-adhd skill enforces ten concrete output-formatting rules through a declarative skill manifest and an evaluation harness that verifies compliance via JSON test cases.**

This open-source skill from the `ayghri/i-have-adhd` repository acts as an **output-shaping plug-in** designed to make LLM responses immediately actionable for readers with ADHD. Its behavior is not hard-coded in procedural logic, but rather defined declaratively in [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) and validated through a data-driven evaluation pipeline that maps test cases directly to the skill's rule set.

---

## Architecture: How Behaviors Are Defined

The i-have-adhd skill follows a **declarative configuration pattern** rather than an imperative implementation. This design allows the behavior to be transparent, version-controlled, and independently testable.

### Core Rule Set in SKILL.md

According to the repository source code, [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) specifies ten concrete formatting rules that every response must obey. These include:

- **Lead with the next action** — no preamble, no context-building
- **Number multi-step tasks** — explicit sequence markers
- **Restate state every turn** — current step position clearly indicated
- **Give specific time estimates** — concrete durations, not vague ranges
- **Make completed work visible** — explicit checkmarks or status indicators
- **Avoid preambles and recaps** — no conversational padding

These rules are **human-readable constraints** that any compatible LLM system must apply when the skill is active. The file path [`skills/i-have-adhd/SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/skills/i-have-adhd/SKILL.md) serves as the single source of truth for expected behavior.

---

## Evaluation Harness: How Behaviors Are Tested

The repository ships a complete **evaluation infrastructure** in the `evals/` directory that transforms the declarative rules into executable test criteria.

### Test Case Definition in cases.jsonl

The file `evals/cases.jsonl` contains the concrete test cases whose **criteria directly map to the skill's rules**. Each case encodes:

- A **prompt** that exercises specific rule combinations
- A **risk level** classification
- A **list of pass/fail criteria** corresponding to SKILL.md rules

For example, the repository source identifies these representative case types:

| Case Name | Rule(s) Tested | Expected Behavior |
|-----------|---------------|-------------------|
| `direct-answer` | Lead with next action, Avoid preambles | Response answers immediately without unnecessary steps |
| `multi-step-progress` | Number tasks, Restate state, Make completion visible | Explicit step counter, current position, and single next action |
| `destructive-action` | Safety override | Blocker flag triggers release-gate failure |

The JSON-L format enables **batch evaluation** across diverse scenario categories while maintaining traceability between rule source and test assertion.

---

## Test Infrastructure: Validating the Evaluation Pipeline

The harness itself is unit-tested in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py), which validates that the evaluation framework correctly applies case criteria to candidate outputs.

### Three Validation Layers

1. **Case catalog integrity** — confirms `cases.jsonl` is well-formed, parseable, and balanced across scenario categories

2. **Scoring logic** — applies weighted dimensions (correctness, autonomy, safety, coherence) and enforces release-gate thresholds

3. **Safety enforcement** — guarantees that blocker-flagged cases (like `destructive-action`) properly fail the release gate regardless of other scores

This **meta-testing approach** ensures that when the harness reports a skill as compliant, the assertion is trustworthy. As implemented in `ayghri/i-have-adhd`, the evaluation framework measures compliance against criteria encoded in cases, which themselves derive from [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) rules.

---

## Skill Invocation and Expected Output Style

The skill activates through standard command syntax in compatible clients.

### Claude-Compatible Invocation

```bash
/i-have-adhd

```

**Multi-step request example** — prompt: *"Plan a safe database schema migration to UUID primary keys"*

```text
1. Back up the current database schema.
2. Add a new UUID column to the users table.
3. Populate the UUID column for existing rows.
4. Update API endpoints to read/write the UUID column.
5. Deploy the changes to staging and run integration tests.

Step 3 of 5 done: back-filling the new column is complete. Next: verify data integrity with `SELECT COUNT(*) FROM users WHERE uuid IS NULL;` (≈ 5 minutes).

```

### Codex Invocation

```bash
$i-have-adhd

```

**Direct-answer example** — prompt: *"What is 17 × 6?"*

```text
Run `echo $((17*6))` → 102.

```

Both examples demonstrate the **compressed, action-first format** that the evaluation cases verify: immediate actionable content, numbered sequences where applicable, explicit state tracking, and zero preamble.

---

## Isolation Mechanism: The Always-On Hook

The [`evals/README.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/README.md) documents a critical testing requirement: an **always-on hook** that can inject the skill into the baseline condition. This ensures that A/B comparisons truly measure the skill's impact rather than confounding variables like prompt engineering or model temperature.

Without this isolation, the evaluation would blur the line between *model capability* and *skill enforcement*. The hook guarantees that measured differences in output format are attributable to the ten rules in [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md).

---

## Key Source Files

| File Path | Purpose | Behavior Link |
|-----------|---------|---------------|
| [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) | Declares ten formatting rules | Source of all tested behaviors |
| `evals/cases.jsonl` | JSON-L test case definitions | Criteria map 1:1 to rules |
| [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) | Unit tests for evaluation harness | Validates scoring and safety logic |
| [`evals/README.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/README.md) | Evaluation architecture documentation | Explains isolation requirements |

---

## Summary

- **The i-have-adhd skill behaviors are defined declaratively** in [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) through ten human-readable formatting rules, not procedural code.

- **Compliance is verified via data-driven evaluation** in `evals/cases.jsonl`, where each test case encodes criteria that map directly to specific rules.

- **The evaluation harness is itself tested** in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py), ensuring integrity of case parsing, weighted scoring, and safety gate enforcement.

- **Skill invocation uses standard commands** (`/i-have-adhd` or `$i-have-adhd`) and produces immediately actionable, numbered, state-tracking output validated by the test suite.

---

## Frequently Asked Questions

### What specific output formatting rules does the i-have-adhd skill enforce?

According to the [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md) source, the skill enforces ten rules including: lead with the next action (no preamble), number multi-step tasks, restate current state every turn, provide specific time estimates, make completed work visible, and eliminate conversational padding like recaps.

### How does the evaluation harness know if a response follows the rules?

The harness compares outputs against **criteria encoded in `evals/cases.jsonl`**. Each criterion directly maps to a rule in [`SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/SKILL.md). For example, the `direct-answer` case checks that responses answer immediately without unnecessary steps, validating the "lead with next action" and "avoid preambles" rules.

### What prevents the evaluation framework itself from having bugs?

The repository includes **meta-tests in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py)** that validate case catalog integrity, scoring mathematics, and safety gate behavior. This ensures that reported compliance scores are trustworthy and that blocker-flagged cases properly fail regardless of other performance dimensions.

### Can the skill be tested in isolation from other system variables?

Yes. According to [`evals/README.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/README.md), the framework supports an **always-on hook** that injects the skill into baseline conditions. This isolation mechanism ensures that A/B comparisons measure pure skill impact rather than confounding factors like prompt variations or model parameters.