Release Process for i-have-adhd: A 7-Step Gate-Controlled Workflow

The i-have-adhd repository uses a gate-controlled release workflow that mandates automated verification, LLM-based evaluation scoring, and manual review before any version can be published.

The ayghri/i-have-adhd project maintains strict quality standards for its ADHD-friendly response system through a rigorous, automated release process. Every change must pass through a multi-stage pipeline that evaluates both correctness and safety against established baselines before merging is permitted. The process is fully defined in CONTRIBUTING.md and enforced through CI workflows in .github/workflows/.

The 7-Step Release Pipeline

The release process for i-have-adhd follows a specific sequence from pull request creation to final publication.

Step 1: Prepare a Pull Request with Proper Labels

Contributors must create a focused PR containing only the intended change and apply mandatory labels as specified in CONTRIBUTING.md. Target labels categorize the change type (Target:Integrations, Target:Evals, Target:Rules, Target:Docs), while Author labels identify the creation method (Author:Human, Author:Hybrid, Author:AI). The PR description must explain what changed, why, the safety impact, and verification steps.

Step 2: Execute the Unit-Test Suite

Before evaluation, the repository's test suite must pass to catch regressions. The command is defined in AGENTS.md:

python3 -m unittest discover -s tests -v

All tests must complete successfully before proceeding to the evaluation phase.

Step 3: Run the Evaluation Harness

The scripts/run_evals.py file serves as the entry point for generating evaluation scores for both baseline and candidate versions. The harness validates case definitions, runs the selected LLM runner, and aggregates scores using subcommands like validate, plan, run, and score. Changes to skills/i-have-adhd/SKILL.md or other core logic must be evaluated against this baseline.

Step 4: Apply the Release Gate

After scoring, the summarize_scores() function (lines 64-68 in scripts/run_evals.py) returns a release_gate object. The gate passes only if three conditions are met:

• No blocking safety or correctness findings exist
• The candidate's correctness and safety scores are not more than 0.1 points lower than the baseline
• The candidate's weighted_score exceeds the baseline

If release_gate.passed is false, the PR must be updated until the gate succeeds.

Step 5: Pass CI Checks

GitHub workflows located in .github/workflows/ (including plugin-load-check.yml and pi-load-check.yml) automatically run unit tests, linting, and release-gate verification for every PR. A PR cannot be merged unless all CI checks succeed.

Step 6: Merge the Pull Request

Once the release gate passes and CI is green, a maintainer merges the PR using the standard GitHub process. CONTRIBUTING.md specifies that a "PR is ready when" these conditions are satisfied.

Step 7: Create a Git Tag and Release

After merging, the maintainer creates an annotated tag (e.g., v1.2.0) and publishes a GitHub release. This tag may trigger downstream publication pipelines such as npm or plugin registries.

Running the Evaluation Harness Locally

Developers can manually trigger the evaluation and scoring process to verify changes before submitting a PR.

To run the evaluation suite for a candidate version:

python3 scripts/run_evals.py run \
  --runner-config evals/runners.example.json \
  --runner openai \
  --condition candidate \
  --condition-skill skills/i-have-adhd/SKILL.md \
  --output evals/results.jsonl

To score the collected results and check the release gate:

python3 scripts/run_evals.py score evals/results.jsonl

The command outputs a JSON object including the release_gate field:

{
  "weights": {...},
  "conditions": {...},
  "release_gate": {"passed": true, "reasons": []}
}

Key Files in the Release Process

The following files define and enforce the release workflow:

• CONTRIBUTING.md – Defines PR labeling requirements, verification steps, and the release-gate policy
• scripts/run_evals.py – Implements the evaluation harness and summarize_scores() logic
• tests/ – Contains the unit-test suite that must pass before merging
• AGENTS.md – Lists the exact CI commands for local verification
• .github/workflows/ – Houses CI pipelines that enforce the release gate
• skills/i-have-adhd/SKILL.md – The canonical skill definition; changes here require full evaluation

Summary

The i-have-adhd release process ensures quality through automated gating:

• Gate-controlled workflow: No release occurs without passing the automated evaluation gate
• Strict scoring thresholds: Correctness and safety scores cannot drop more than 0.1 points below baseline, and weighted scores must improve
• Mandatory labeling: All PRs require Target and Author labels per CONTRIBUTING.md
• CI enforcement: GitHub workflows in .github/workflows/ block merges that fail unit tests or the release gate
• Manual finalization: Maintainers create annotated Git tags only after all automated checks pass

Frequently Asked Questions

What causes the release gate to fail?

The release gate fails if scripts/run_evals.py detects blocking safety or correctness findings, if correctness or safety scores drop more than 0.1 points below the baseline, or if the candidate's weighted_score does not exceed the baseline. The summarize_scores() function enforces these thresholds at lines 64-68.

How do I run the evaluation harness locally?

Execute python3 scripts/run_evals.py run with your runner configuration and skill path, followed by python3 scripts/run_evals.py score on the output file. See AGENTS.md for the exact command syntax and CONTRIBUTING.md for prerequisites.

What labels are required for a release pull request?

Every PR must include a Target label (such as Target:Integrations or Target:Rules) and an Author label (Author:Human, Author:Hybrid, or Author:AI). These are defined in CONTRIBUTING.md and are enforced by CI checks.

Where is the release gate logic implemented?

The release gate logic resides in the summarize_scores() function within scripts/run_evals.py, specifically at lines 64-68. This function calculates whether the candidate version meets the safety, correctness, and weighted score thresholds required for publication.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}