i-Have-ADHD Skill Behaviors Tested: Rules, Evaluation Harness, and Compliance Validation

The i-have-adhd skill enforces ten concrete output-formatting rules through a declarative skill manifest and an evaluation harness that verifies compliance via JSON test cases.

This open-source skill from the ayghri/i-have-adhd repository acts as an output-shaping plug-in designed to make LLM responses immediately actionable for readers with ADHD. Its behavior is not hard-coded in procedural logic, but rather defined declaratively in SKILL.md and validated through a data-driven evaluation pipeline that maps test cases directly to the skill's rule set.

---

Architecture: How Behaviors Are Defined

The i-have-adhd skill follows a declarative configuration pattern rather than an imperative implementation. This design allows the behavior to be transparent, version-controlled, and independently testable.

Core Rule Set in SKILL.md

According to the repository source code, SKILL.md specifies ten concrete formatting rules that every response must obey. These include:

• Lead with the next action — no preamble, no context-building
• Number multi-step tasks — explicit sequence markers
• Restate state every turn — current step position clearly indicated
• Give specific time estimates — concrete durations, not vague ranges
• Make completed work visible — explicit checkmarks or status indicators
• Avoid preambles and recaps — no conversational padding

These rules are human-readable constraints that any compatible LLM system must apply when the skill is active. The file path skills/i-have-adhd/SKILL.md serves as the single source of truth for expected behavior.

---

Evaluation Harness: How Behaviors Are Tested

The repository ships a complete evaluation infrastructure in the evals/ directory that transforms the declarative rules into executable test criteria.

Test Case Definition in cases.jsonl

The file evals/cases.jsonl contains the concrete test cases whose criteria directly map to the skill's rules. Each case encodes:

• A prompt that exercises specific rule combinations
• A risk level classification
• A list of pass/fail criteria corresponding to SKILL.md rules

For example, the repository source identifies these representative case types:

Case Name	Rule(s) Tested	Expected Behavior
direct-answer	Lead with next action, Avoid preambles	Response answers immediately without unnecessary steps
multi-step-progress	Number tasks, Restate state, Make completion visible	Explicit step counter, current position, and single next action
destructive-action	Safety override	Blocker flag triggers release-gate failure

The JSON-L format enables batch evaluation across diverse scenario categories while maintaining traceability between rule source and test assertion.

---

Test Infrastructure: Validating the Evaluation Pipeline

The harness itself is unit-tested in tests/test_run_evals.py, which validates that the evaluation framework correctly applies case criteria to candidate outputs.

Three Validation Layers

1. Case catalog integrity — confirms cases.jsonl is well-formed, parseable, and balanced across scenario categories

2. Scoring logic — applies weighted dimensions (correctness, autonomy, safety, coherence) and enforces release-gate thresholds

3. Safety enforcement — guarantees that blocker-flagged cases (like destructive-action) properly fail the release gate regardless of other scores

This meta-testing approach ensures that when the harness reports a skill as compliant, the assertion is trustworthy. As implemented in ayghri/i-have-adhd, the evaluation framework measures compliance against criteria encoded in cases, which themselves derive from SKILL.md rules.

---

Skill Invocation and Expected Output Style

The skill activates through standard command syntax in compatible clients.

Claude-Compatible Invocation

/i-have-adhd

Multi-step request example — prompt: "Plan a safe database schema migration to UUID primary keys"

1. Back up the current database schema.
2. Add a new UUID column to the users table.
3. Populate the UUID column for existing rows.
4. Update API endpoints to read/write the UUID column.
5. Deploy the changes to staging and run integration tests.

Step 3 of 5 done: back-filling the new column is complete. Next: verify data integrity with `SELECT COUNT(*) FROM users WHERE uuid IS NULL;` (≈ 5 minutes).

Codex Invocation

$i-have-adhd

Direct-answer example — prompt: "What is 17 × 6?"

Run `echo $((17*6))` → 102.

Both examples demonstrate the compressed, action-first format that the evaluation cases verify: immediate actionable content, numbered sequences where applicable, explicit state tracking, and zero preamble.

---

Isolation Mechanism: The Always-On Hook

The evals/README.md documents a critical testing requirement: an always-on hook that can inject the skill into the baseline condition. This ensures that A/B comparisons truly measure the skill's impact rather than confounding variables like prompt engineering or model temperature.

Without this isolation, the evaluation would blur the line between model capability and skill enforcement. The hook guarantees that measured differences in output format are attributable to the ten rules in SKILL.md.

---

Key Source Files

File Path	Purpose	Behavior Link
SKILL.md	Declares ten formatting rules	Source of all tested behaviors
evals/cases.jsonl	JSON-L test case definitions	Criteria map 1:1 to rules
tests/test_run_evals.py	Unit tests for evaluation harness	Validates scoring and safety logic
evals/README.md	Evaluation architecture documentation	Explains isolation requirements

---

Summary

• The i-have-adhd skill behaviors are defined declaratively in SKILL.md through ten human-readable formatting rules, not procedural code.

• Compliance is verified via data-driven evaluation in evals/cases.jsonl, where each test case encodes criteria that map directly to specific rules.

• The evaluation harness is itself tested in tests/test_run_evals.py, ensuring integrity of case parsing, weighted scoring, and safety gate enforcement.

• Skill invocation uses standard commands (/i-have-adhd or $i-have-adhd) and produces immediately actionable, numbered, state-tracking output validated by the test suite.

---

Frequently Asked Questions

What specific output formatting rules does the i-have-adhd skill enforce?

According to the SKILL.md source, the skill enforces ten rules including: lead with the next action (no preamble), number multi-step tasks, restate current state every turn, provide specific time estimates, make completed work visible, and eliminate conversational padding like recaps.

How does the evaluation harness know if a response follows the rules?

The harness compares outputs against criteria encoded in evals/cases.jsonl. Each criterion directly maps to a rule in SKILL.md. For example, the direct-answer case checks that responses answer immediately without unnecessary steps, validating the "lead with next action" and "avoid preambles" rules.

What prevents the evaluation framework itself from having bugs?

The repository includes meta-tests in tests/test_run_evals.py that validate case catalog integrity, scoring mathematics, and safety gate behavior. This ensures that reported compliance scores are trustworthy and that blocker-flagged cases properly fail regardless of other performance dimensions.

Can the skill be tested in isolation from other system variables?

Yes. According to evals/README.md, the framework supports an always-on hook that injects the skill into baseline conditions. This isolation mechanism ensures that A/B comparisons measure pure skill impact rather than confounding factors like prompt variations or model parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}