# How Is Correctness Weighted in the i-have-adhd Evaluation Rubric?

> Discover how correctness is weighted in the i-have-adhd evaluation rubric. This key criterion accounts for 35% of the total score in the ayghri/i-have-adhd repository.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: deep-dive
- Published: 2026-08-01

---

**In the `ayghri/i-have-adhd` repository, the Correctness dimension carries a weight of 35% in the evaluation rubric, making it the single most heavily weighted criterion.**

The `i-have-adhd` repository implements a structured evaluation system for assessing model outputs across multiple quality dimensions. Understanding how Correctness is weighted helps interpret evaluation scores and prioritize improvements in model behavior.

## The 35% Weight for Correctness in the Rubric

The evaluation rubric explicitly defines dimension weights in [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md). Correctness receives the highest allocation at **35%**, reflecting its priority as the foundational quality of any model response.

According to the rubric, Correctness measures **factual and technical accuracy** plus whether **required details are preserved** in the output. This emphasis ensures that even well-formatted, concise responses cannot score highly if they contain factual errors or omit critical information.

## How Correctness Weight Is Implemented in the Scoring Script

The weight is programmatically enforced in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) through the `WEIGHTS` dictionary:

```python
WEIGHTS = {
    "correctness": 0.35,
    "autonomy": 0.25,
    "actionability": 0.20,
    "safety": 0.10,
    "concision": 0.10,
}

```

This dictionary maps each dimension name to its decimal weight. When [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) computes final scores, it multiplies each dimension's rating (scored 1–5) by its corresponding weight and sums the results.

### Verifying the Correctness Weight Programmatically

You can extract and verify the Correctness weight directly from the source:

```python
from pathlib import Path

# Path to the evaluation script relative to repository root

script_path = Path("scripts/run_evals.py")

# Parse the WEIGHTS dictionary (simplified extraction)

weights = {}
for line in script_path.read_text().splitlines():
    if line.strip().startswith('"correctness":'):
        weight = float(line.split(":")[1].strip().strip(","))
        weights["correctness"] = weight
        break

print(f"Correctness weight = {weights['correctness'] * 100}%")

# → Correctness weight = 35.0%

```

This confirms the 35% allocation matches the rubric documentation.

## Complete Weight Distribution Across All Dimensions

The full weighting scheme prioritizes accuracy and self-sufficiency:

| Dimension | Weight | Purpose |
|-----------|--------|---------|
| **Correctness** | **35%** | Factual accuracy and detail preservation |
| Autonomy | 25% | Self-directed problem-solving without unnecessary clarification |
| Actionability | 20% | Practical, implementable guidance |
| Safety | 10% | Harm avoidance and appropriate boundaries |
| Concision | 10% | Brevity without sacrificing completeness |

The **10-percentage-point gap** between Correctness (35%) and Autonomy (25%) demonstrates that technical accuracy takes precedence over independence in this evaluation framework. Even a perfectly autonomous response cannot compensate for correctness failures.

## Impact on Aggregate Scoring

The weighting algorithm in [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) applies these proportions uniformly across evaluation conditions (baseline vs. candidate comparisons). A Correctness score of 5 contributes **1.75 points** to the weighted total (5 × 0.35), while a Concision score of 5 contributes only **0.50 points** (5 × 0.10).

This mathematical structure means Correctness improvements yield **3.5× more score impact** than equivalent improvements in Safety or Concision.

## Summary

- **Correctness is weighted at 35%** in the `i-have-adhd` evaluation rubric, defined in [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md).
- The same 35% weight appears in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) as `"correctness": 0.35` in the `WEIGHTS` dictionary.
- This makes Correctness the highest-weighted dimension, **10 points above Autonomy** (25%) and **25 points above Safety/Concision** (10% each).
- Final scores multiply raw ratings (1–5) by these weights, so Correctness dominates aggregate results.

## Frequently Asked Questions

### What does the Correctness dimension actually measure in i-have-adhd?

Correctness evaluates **factual and technical accuracy** plus whether the output preserves all **required details** from the input or context. A response can be well-structured and safe yet score poorly on Correctness if it contains technical errors or omits critical information.

### Why is Correctness weighted higher than Autonomy?

The 35% vs. 25% allocation reflects a prioritization hierarchy where **accurate information outweighs self-directed behavior**. According to the rubric structure in [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md), a model that asks clarifying questions but eventually provides correct answers scores better than one that proceeds independently but produces incorrect outputs.

### Can I modify the Correctness weight in my own evaluation runs?

Yes. Edit the `WEIGHTS` dictionary in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) to adjust the `0.35` value, or modify the table in [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md) to document your changes. The scoring script reads weights at runtime, so no recompilation is required.