# How Frequently Are the Evaluations for 'i-have-adhd' Updated?

> Discover how frequently the i-have-adhd repository evaluations are updated. Learn about the manual update process and why there's no automated schedule.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: faq
- Published: 2026-08-03

---

**Evaluations for the *i-have-adhd* repository are updated only when maintainers manually trigger them via [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py)—there is no automated schedule, cron job, or CI pipeline that refreshes them.**

The *i-have-adhd* project includes a complete evaluation harness in the `evals/` directory for benchmarking AI-assisted ADHD coaching responses. Understanding how often these evaluations update helps contributors and researchers know when test results reflect the latest code changes. According to the repository source code, evaluation updates follow a **manual, intention-driven workflow** rather than any fixed cadence.

## Manual Triggering via run_evals.py

All evaluation operations flow through **[`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py)**, a command-line tool that handles validation, planning, execution, and scoring. The script is **not invoked automatically**—maintainers run it explicitly when they want fresh results.

The script supports four primary commands:

```bash

# Validate the evaluation definition

python3 scripts/run_evals.py validate

# Generate an execution plan

python3 scripts/run_evals.py plan --trials 3 --include-comparator

# Execute a full evaluation run

python3 scripts/run_evals.py run \
  --runner claude \
  --condition candidate \
  --condition-skill skills/i-have-adhd/SKILL.md \
  --trials 3 \
  --budget-usd 12.50 \
  --output evals/results/responses.jsonl

# Score the collected responses

python3 scripts/run_evals.py score evals/results/scores.jsonl

```

Because each command requires explicit invocation, the frequency of evaluation updates depends entirely on **maintainer discretion**.

## What Triggers an Evaluation Update

Updates typically occur in these scenarios:

- **After code changes** to the skill definition in [`skills/i-have-adhd/SKILL.md`](https://github.com/ayghri/i-have-adhd/blob/main/skills/i-have-adhd/SKILL.md)
- **When benchmarking a new model version** or different runner (e.g., switching from Claude to another provider)
- **Prior to releases** to validate performance against the rubric
- **Ad hoc comparisons** between baseline and candidate conditions

There is no evidence in the repository of GitHub Actions workflows, cron schedules, or other automation that would trigger [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) without human intervention.

## Static Evaluation Assets

The underlying test cases and scoring criteria remain stable unless manually edited. Key files include:

- **`evals/cases.jsonl`** — JSONL file containing all evaluation cases (test scenarios)
- **[`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md)** — Scoring rubric defining dimensions, weights, and blocking criteria
- **[`evals/README.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/README.md)** — Documentation for the evaluation workflow

These files are version-controlled but **not regenerated automatically**. Only the results produced by [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) change with each manual execution.

## Complete Evaluation Workflow Example

To perform a full baseline-versus-candidate comparison that updates stored results:

```bash

# 1. Validate definitions

python3 scripts/run_evals.py validate

# 2. Run both conditions

python3 scripts/run_evals.py run \
  --runner claude \
  --condition baseline \
  --condition candidate \
  --condition-skill skills/i-have-adhd/SKILL.md \
  --trials 3 \
  --budget-usd 12.50 \
  --output evals/results/responses.jsonl

# 3. Generate scores

python3 scripts/run_evals.py score evals/results/scores.jsonl

```

Each step requires deliberate execution. The resulting `responses.jsonl` and score files represent the **latest intentional evaluation**—not a continuously updated dataset.

## Summary

- **No automation**: Evaluations lack cron jobs, CI triggers, or scheduled pipelines
- **Manual execution**: [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) must be invoked explicitly by maintainers
- **Trigger events**: Code changes, new skill versions, or model benchmarking needs
- **Static inputs**: `evals/cases.jsonl`, [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md), and related files change only through commits
- **Result pacing**: Update frequency equals how often maintainers choose to run evaluations

## Frequently Asked Questions

### Is there a daily or weekly schedule for running i-have-adhd evaluations?

No. The repository contains no automated scheduling mechanism. Evaluations update only when someone manually executes [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py).

### How can I check when the most recent evaluation was performed?

Examine the commit history of `evals/results/` files or any timestamped outputs from previous [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) executions. The repository does not expose a dedicated "last run" endpoint.

### Can I trigger my own evaluation run as an external contributor?

Yes. Clone the repository, ensure you have API credentials for your chosen runner (e.g., Claude), and execute the [`run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/run_evals.py) commands locally with your own budget parameters.

### What does it cost to run a full evaluation suite?

The example commands specify `--budget-usd 12.50`, suggesting a typical run costs approximately $10–15 depending on trial count and model pricing. Adjust `--budget-usd` and `--trials` to control expenses.