# How Edge Cases Are Handled and Tested in the *i-have-adhd* Repository

> Discover how i-have-adhd handles edge cases with input validation, fallback logic, and extensive testing for malformed inputs, API failures, and config errors. Learn more.

- Repository: [Ayoub Ghriss/i-have-adhd](https://github.com/ayghri/i-have-adhd)
- Tags: testing
- Published: 2026-08-03

---

**The *i-have-adhd* project handles edge cases through robust input validation, model-agnostic fallback logic, and a comprehensive test suite in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) that deliberately exercises malformed inputs, API failures, and configuration errors.**

This open-source evaluation runner processes LLM test cases via a command-line interface in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py). The codebase demonstrates defensive programming patterns designed to survive real-world deployment conditions—including bad data, missing credentials, and network instability. This article examines exactly how edge cases are managed and verified according to the `ayghri/i-have-adhd` source code.

---

## Input Validation and Malformed Data Handling

The runner's first line of defense operates in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py), where every entry from `evals/cases.jsonl` undergoes strict validation.

### Required Field Verification

Each case must contain `prompt`, `expected`, and `model` keys. The `validate_case()` function raises an explicit `ValueError` when any required field is absent, preventing downstream failures from cryptic stack traces.

```python

# From tests/test_run_evals.py — validates missing field detection

def test_missing_prompt_raises():
    malformed_case = {"expected": "Yes", "model": "openai"}  # `prompt` omitted

    with pytest.raises(ValueError, match="Missing required field 'prompt'"):
        validate_case(malformed_case)

```

### Input Size and Format Normalization

The runner implements two additional protective layers:

- **Token limit enforcement** — Prompts exceeding model context windows are truncated or padded before API submission
- **Whitespace normalization** — OS-specific line endings (`\r\n` vs `\n`) and surrounding whitespace are standardized so platform differences don't cause false negatives

These transformations occur before any external API call, ensuring consistent behavior across operating systems.

---

## Model Provider Flexibility and Graceful Degradation

The architecture supports dual backend providers through configuration files in `skills/i-have-adhd/agents/`:

| Provider | Configuration File |
|----------|------------------|
| OpenAI | [`agents/openai.yaml`](https://github.com/ayghri/i-have-adhd/blob/main/agents/openai.yaml) |
| Gemini | [`agents/gemini.toml`](https://github.com/ayghri/i-have-adhd/blob/main/agents/gemini.toml) |

When [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) detects that no API keys are configured for either provider, it falls back to a no-op stub rather than crashing. This behavior emits a clear warning and allows the evaluation batch to continue—a critical edge case for CI environments or offline development.

```bash

# Run with fail-fast disabled to continue through provider unavailability

python scripts/run_evals.py \
    --cases evals/cases.jsonl \
    --rubric evals/rubric.md

```

```python

# Programmatic invocation with explicit error tolerance

from scripts.run_evals import run_evals

run_evals(
    cases_path="evals/cases.jsonl",
    rubric_path="evals/rubric.md",
    model="openai",          # or "gemini"

    fail_fast=False         # continue after errors

)

```

---

## Result Verification with Fuzzy Matching

Post-execution validation in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) applies the scoring rubric defined in [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md). The comparison logic intentionally tolerates:

- Case variations (`"Yes"` vs `"yes"`)
- Extraneous punctuation
- Leading/trailing whitespace

These are treated as acceptable variations **unless** the rubric explicitly flags them as critical distinctions. This prevents brittle exact-string matching from producing false failures on semantically equivalent outputs.

---

## Network and API Error Recovery

All external calls to LLM providers are wrapped in comprehensive `try/except` blocks capturing:

- Network timeouts
- Connection failures
- API-level exceptions (rate limits, invalid requests, server errors)

Rather than aborting the entire batch, the runner logs a concise error message and records the specific test as "failed," preserving partial results and execution continuity.

---

## Comprehensive Edge Case Test Coverage

The test suite in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) systematically validates every defensive code path through deliberately malformed inputs:

| Test Category | Input Condition | Expected Behavior |
|-------------|---------------|-------------------|
| **Missing fields** | Case without `prompt` key | `ValueError` with clear message |
| **Oversized input** | Prompt exceeding token limit | Silent truncation, no exception |
| **Invalid JSON** | Malformed line in `cases.jsonl` | Skip line with logged warning |
| **Unavailable model** | No API keys in environment | Warning emission, continue execution |

These tests execute automatically via the CI workflow defined in [`.github/workflows/plugin-load-check.yml`](https://github.com/ayghri/i-have-adhd/blob/main/.github/workflows/plugin-load-check.yml), ensuring regressions in edge-case handling are caught before release.

---

## Summary

- **Strict validation** of required fields (`prompt`, `expected`, `model`) with explicit `ValueError` exceptions
- **Input normalization** for token limits and cross-platform whitespace consistency
- **Provider-agnostic fallback** to no-op stubs when API credentials are unavailable
- **Fuzzy result matching** against [`evals/rubric.md`](https://github.com/ayghri/i-have-adhd/blob/main/evals/rubric.md) with configurable strictness
- **Graceful degradation** on network failures—log and continue rather than crash
- **Systematic test coverage** in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) exercising all failure modes

---

## Frequently Asked Questions

### What happens if a test case is missing the required `prompt` field?

The `validate_case()` function in [`scripts/run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/scripts/run_evals.py) raises a `ValueError` with the message "Missing required field 'prompt'". This is verified by `test_missing_prompt_raises()` in the test suite.

### How does the runner handle prompts that exceed the model's context window?

Input strings are automatically truncated or padded to fit within token limits before API submission. The `test_oversized_input` case confirms this occurs without raising exceptions.

### Can the evaluation suite run without API credentials configured?

Yes. When neither OpenAI nor Gemini credentials are present, the runner falls back to a no-op stub, emits a warning, and continues processing. This is tested via environment simulation in [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py).

### Where are the edge-case tests executed in CI?

All tests run through the workflow defined in [`.github/workflows/plugin-load-check.yml`](https://github.com/ayghri/i-have-adhd/blob/main/.github/workflows/plugin-load-check.yml), which executes [`tests/test_run_evals.py`](https://github.com/ayghri/i-have-adhd/blob/main/tests/test_run_evals.py) against the full matrix of malformed inputs and failure conditions.