# How the 173-Case Routing Regression Benchmark Ensures Routing Accuracy in reverse-skill

> Ensure routing accuracy with the 173-case routing regression benchmark. Discover how this method validates natural language hints against expected skill identifiers for 100% test pass rates before merges.

- Repository: [ZhaoXu/reverse-skill](https://github.com/zhaoxuya520/reverse-skill)
- Tags: deep-dive
- Published: 2026-09-01

---

**The 173-case routing regression benchmark guarantees routing accuracy by requiring 100% test pass rate before any merge, with each case validating that a natural-language hint resolves to an exact expected skill identifier.**

The *reverse-skill* repository maintains rigorous routing accuracy through a comprehensive regression test suite. This benchmark, stored as a JSON file with 173 test cases, serves as the definitive validation mechanism for the routing engine that maps user hints to skill modules.

## The Core Benchmark Structure

The routing regression benchmark lives in [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json). Each entry represents a **test case** containing three fields:

- `hint` — the natural-language input that triggers routing
- `expect` — the skill identifier that must be returned
- `quick` — a boolean flag for test categorization

For the 73rd case (I73), the benchmark entry appears as:

```json
{
  "hint": "some hint text …",
  "expect": "R7",
  "quick": false
}

```

This structure ensures every routing decision has a documented, verifiable expected outcome.

## The Four-Stage Validation Process

### Stage 1: Case Loading

The test runner loads the benchmark and extracts individual cases by index. In `skills/scripts/test-routing.ps1` and its Bash equivalent [`skills/scripts/test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/test-routing.sh), the runner iterates through all 173 entries sequentially.

### Stage 2: Router Execution

The routing engine receives the hint and consults [`skills/config/routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/config/routing.json) — the authoritative routing table that maps hint patterns to skill IDs. The engine applies its matching logic and returns the resolved identifier.

### Stage 3: Result Validation

The runner performs an exact string comparison between the returned skill ID and the case's `expect` field. Any mismatch causes immediate benchmark failure, blocking the CI pipeline.

### Stage 4: Cross-Validation via Coherence Checks

The `skills/scripts/verify-routing-coherence.ps1` script enforces additional integrity constraints:

- **Minimum case count**: Benchmark must contain at least 100 cases
- **Well-formed identifiers**: All `expect` values follow valid format rules
- **Ghost prevention**: Every `expect` ID must exist in [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json)

These checks prevent orphaned test cases and ensure benchmark maintainability.

## Running the Benchmark

### Windows (PowerShell)

```powershell

# From repository root

powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1

```

### Linux/macOS (Bash)

```bash
bash skills/scripts/test-routing.sh

```

### Programmatic Verification (Python)

This snippet demonstrates the validation logic used by the official runners:

```python
import json
import subprocess
import pathlib

benchmark_path = pathlib.Path('skills/tests/routing-benchmark.json')
benchmark = json.loads(benchmark_path.read_text(encoding='utf-8-sig'))

case = benchmark['cases'][72]          # 73rd case (0-based index)

hint = case['hint']
expected = case['expect']

# Invoke PowerShell router

result = subprocess.check_output([
    'powershell', '-NoProfile', '-Command',
    f'& {{ .\\skills\\scripts\\router.ps1 -Hint \"{hint}\" }}'
]).decode().strip()

assert result == expected, f'Routing mismatch: {result} ≠ {expected}'
print('I73 case passed')

```

## Key Files in the Verification Pipeline

| File | Purpose |
|------|---------|
| [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json) | 173-case test suite including I73 |
| `skills/scripts/test-routing.ps1` | PowerShell benchmark runner |
| [`skills/scripts/test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/test-routing.sh) | Bash benchmark runner |
| `skills/scripts/verify-routing-coherence.ps1` | Sanity and integrity validator |
| [`skills/config/routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/config/routing.json) | Authoritative routing table |

## Summary

- The **173-case routing regression benchmark** serves as the single source of truth for routing correctness in *reverse-skill*
- Each case validates exact matching between natural-language hints and skill identifiers
- **Four-stage validation** (load, execute, compare, cross-check) eliminates routing errors
- **100% pass rate requirement** prevents regressions from reaching production
- Coherence verification in `verify-routing-coherence.ps1` maintains benchmark integrity against schema drift

## Frequently Asked Questions

### What makes the 173-case benchmark a "regression" test?

A regression test detects unintended changes to previously working behavior. The 173-case benchmark captures 173 verified correct routing decisions as permanent fixtures. Any code change that alters routing logic must preserve all 173 outcomes, ensuring that fixes don't break existing functionality.

### How does the I73 case specifically test routing accuracy?

The I73 case (73rd entry, index 72) validates that a specific natural-language hint resolves to skill identifier `R7`. This case, like all others, documents a real user scenario where a particular hint pattern must map to a predetermined skill module. Failure indicates the routing engine no longer recognizes that hint pattern correctly.

### What happens if the benchmark fails during CI?

The repository blocks merge. Per the verification pipeline design, `test-routing.ps1` or [`test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/test-routing.sh) returns a non-zero exit code on any mismatch. CI systems interpret this as a failed job, preventing the pull request from merging until the routing logic or benchmark data is corrected.

### Can the benchmark detect missing or obsolete skill mappings?

Yes. The `verify-routing-coherence.ps1` script explicitly checks that every `expect` ID in the benchmark exists in [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json). This "ghost prevention" ensures test cases always reference valid skills, catching deletions or renames that would otherwise orphan test expectations.