# i-Have-ADHD Skill Performance Metrics: Complete Evaluation Framework Explained

> Discover the i-have-adhd skill performance metrics. Learn about correctness autonomy actionability safety and concision in this comprehensive evaluation framework.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: performance
- Published: 2026-08-03

---

**The i-have-adhd skill uses five weighted metrics—correctness (35%), autonomy (25%), actionability (20%), safety (10%), and concision (10%)—to calculate an overall weighted score for performance evaluation.**

The `ayghri/i-have-adhd` repository implements a structured evaluation framework for measuring how effectively the skill assists users with ADHD. This framework, defined in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py), combines domain-specific quality criteria with automated scoring to produce reproducible performance reports and release gate decisions.

## The Five Core Performance Metrics

The evaluation system captures five distinct dimensions of response quality. Each metric is scored on a **1–5 scale** and contributes to a final weighted score.

### Correctness (Weight: 0.35)

The **correctness** metric measures factual accuracy and relevance to the user's query. At 35% of the total weight, this is the dominant factor in performance assessment. Evaluators assign scores based on whether the response correctly addresses ADHD-related challenges with accurate information.

### Autonomy (Weight: 0.25)

**Autonomy** evaluates how much the response reduces cognitive effort for the user—critical for an ADHD-focused tool. Higher scores indicate the skill handled complexity internally rather than burdening the user with decisions or follow-up requirements.

### Actionability (Weight: 0.20)

The **actionability** metric scores the concreteness of next steps provided. Responses that include specific, timestamped, or immediately implementable instructions score higher than vague suggestions.

### Safety (Weight: 0.10)

**Safety** checks for harmful advice regarding medication, unverified treatments, or other risky recommendations. While weighted lower, this metric serves as a hard blocker if critical failures occur.

### Concision (Weight: 0.10)

**Concision** rewards brevity that preserves necessary detail. For ADHD users, this balances completeness against the cognitive cost of lengthy responses.

## How the Weighted Score Is Calculated

The metric definitions and their numeric weights are declared in the `WEIGHTS` constant in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) (lines 20–25):

```python

# From scripts/run_evals.py

WEIGHTS = {
    "correctness": 0.35,
    "autonomy": 0.25,
    "actionability": 0.20,
    "safety": 0.10,
    "concision": 0.10,
}

```

After validation via `_validate_score` (lines 90–92), which enforces the 1–5 range, the `summarize_scores` function (lines 64–68) computes averages and applies the weighted formula:

```

weighted_score = Σ(metric_average × weight)

```

The result is combined with a release gate pass/fail status in the final report structure.

## Running the Evaluation

### Command-Line Execution

```bash
python scripts/run_evals.py --cases evals/cases.jsonl --output results.json

```

### Programmatic Score Analysis

```python
from scripts.run_evals import summarize_scores

# `scores` is a list of dicts loaded from a JSON-Lines file

final_report = summarize_scores(scores)

print(final_report["conditions"]["candidate"]["weighted_score"])
print(final_report["release_gate"]["passed"])

```

## Evaluation Data Format

Each evaluation row follows a structured schema. Here is a valid example:

```json
{
  "case_id": "example-1",
  "trial": 1,
  "condition": "candidate",
  "correctness": 4,
  "autonomy": 5,
  "actionability": 3,
  "safety": 5,
  "concision": 4,
  "blocker": false,
  "notes": "All criteria met."
}

```

The **condition** field distinguishes between *baseline* and *candidate* versions for A/B comparisons. The **blocker** boolean can override the automated release gate if manual review identifies critical issues.

## Key Source Files

- **[`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py)** – Core evaluation logic, metric definitions (`WEIGHTS`), validation (`_validate_score`), and reporting (`summarize_scores`)

- **[`skills/i-have-adhd/SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/skills/i-have-adhd/SKILL.md)** – Behavioral specification providing context for metric design decisions

## Summary

- **Five weighted metrics** define i-have-adhd skill performance: correctness, autonomy, actionability, safety, and concision

- **Correctness dominates** at 35% weight, reflecting prioritization of accurate ADHD-related guidance

- **Score range is fixed** at 1–5 per metric, enforced by `_validate_score` in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py)

- **Weighted aggregation** produces the final score used for release gate decisions

- **Structured JSON-Lines format** enables batch evaluation and automated comparison between skill versions

## Frequently Asked Questions

### What is the most important metric for the i-have-adhd skill?

**Correctness carries the highest weight at 35%**. This reflects the priority of accurate information when assisting users with ADHD, where misinformation about symptoms, treatments, or coping strategies could be particularly harmful.

### How is the final score calculated from individual metrics?

The `summarize_scores` function in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) first computes the average of each metric across all evaluation rows, then applies the formula: **weighted_score = Σ(metric_average × weight)**. This aggregated value determines whether the release gate passes.

### Can an evaluation fail even if the weighted score is high?

Yes. The **blocker** field in evaluation rows allows manual override of the automated release gate. Additionally, the `_validate_score` function rejects any scores outside the 1–5 range, preventing calculation with invalid data.

### Where are the metric weights defined in the source code?

The `WEIGHTS` dictionary is defined at **lines 20–25 of [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py)**. Modifying this constant changes how individual metrics contribute to the final weighted score for all subsequent evaluations.