# How Regression Testing in reverse-skill Validates Routing Logic Automatically

> Learn how reverse-skill performs automated regression testing. A benchmark suite validates routing logic, ensuring accuracy through PowerShell and Bash harnesses. Discover the process now.

- Repository: [ZhaoXu/reverse-skill](https://github.com/zhaoxuya520/reverse-skill)
- Tags: how-to-guide
- Published: 2026-08-23

---

**Regression testing in reverse-skill is performed through an automated benchmark suite defined in [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json) and executed by cross-platform PowerShell and Bash harnesses that verify routing output against expected skill identifiers.**

The `zhaoxuya520/reverse-skill` repository maintains routing integrity through deterministic regression testing. Every modification to [`skills/config/routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/config/routing.json) or individual skill scripts must satisfy the entire benchmark catalogue to prevent regressions. The framework exercises both Windows and Unix-like platforms through dedicated test harnesses that isolate execution environments and validate output formats.

## The Regression Test Architecture

### Benchmark Definition in routing-benchmark.json

The canonical test catalogue lives at [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json). This JSON file enumerates every routing scenario as an object containing three fields:

- **`hint`** – The user-level query string passed to the router
- **`expect`** – The primary skill identifier expected in the output
- **`quick`** – A boolean flag marking the case as part of the minimal smoke-test subset

When developers append new routing logic, they extend the `cases` array in this file. The next automated run will automatically validate the new scenario without requiring changes to the test harness logic.

### Cross-Platform Test Harnesses

Two scripts provide identical regression coverage across operating systems:

1. **`skills/scripts/test-routing.ps1`** – PowerShell implementation for Windows environments
2. **[`skills/scripts/test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/test-routing.sh)** – Bash implementation for Linux and macOS

Both harnesses perform the same core operations: they spawn isolated temporary directories, invoke the master router with benchmark hints, parse the generated [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md), and compare extracted skill IDs against expected values.

## How the Test Harnesses Execute Routing Scenarios

### PowerShell Implementation

`skills/scripts/test-routing.ps1` orchestrates the Windows regression pipeline through the following steps:

1. Loads the benchmark JSON and iterates over the `cases` array
2. Creates a temporary directory for each test case to ensure isolation
3. Invokes `skills/scripts/master-route.ps1` via the host executable with the `--hint` parameter
4. Parses [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md) using `skills/scripts/lib/RouteScope.ps1` to extract the `Id` field
5. Compares the extracted ID against the `expect` value from the benchmark
6. Accumulates PASS/FAIL counts and writes a summary log to the designated directory

The script supports a `-Quick` switch that filters execution to only cases where `"quick": true`, enabling rapid smoke testing during development.

```powershell

# Run the full regression suite on Windows

powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1

# Execute only the minimal smoke-test set

powershell -File skills/scripts/test-routing.ps1 -Quick

```

### Bash Implementation

[`skills/scripts/test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/test-routing.sh) provides equivalent functionality for Unix platforms with additional path-resolution validation:

1. Determines the available Python interpreter (lines 10-15)
2. Embeds a Python snippet to parse the JSON benchmark
3. Executes [`skills/scripts/master-route.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/master-route.sh) for each case with appropriate `--hint` arguments
4. Inspects [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md) for the `- primary:` skill identifier
5. Performs exit-code validation and aggregates results

The Bash version includes a specific regression check absent from the PowerShell implementation: it verifies that the router writes output to the caller’s project directory when `--out-dir` is omitted, ensuring the tool does not pollute the installed package directory.

```bash

# Run the full regression suite on Linux/macOS

bash skills/scripts/test-routing.sh

```

## Special Regression Checks

### Default-Root Path Validation

Beyond per-case routing validation, the Bash test harness implements a **default-root regression** check (lines 70-78). This test ensures that when [`master-route.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/master-route.sh) runs without an explicit `--out-dir` argument, the generated [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md) appears within the caller’s project workspace rather than the reverse-skill installation directory.

This guard prevents subtle path-resolution bugs where the router might incorrectly resolve output locations relative to its own source tree instead of the user’s working directory. The test explicitly verifies file existence in the temporary directory after execution completes.

## Extending the Regression Suite

Adding new routing scenarios requires only modifying the benchmark JSON:

```json
{
  "hint": "new reverse-engineering scenario",
  "expect": "R99",
  "quick": true
}

```

Append this object to the `cases` array in [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json). Both harnesses will automatically incorporate the new case in subsequent runs, validating that the routing logic correctly maps the new hint to skill `R99`.

## Summary

- **Regression testing in reverse-skill** relies on [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json) as the single source of truth for routing scenarios.
- Cross-platform harnesses at `skills/scripts/test-routing.ps1` and [`skills/scripts/test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/test-routing.sh) execute the benchmark deterministically on Windows and Unix.
- Each test run creates isolated temporary directories, invokes the master router, and parses [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md) to verify the primary skill identifier.
- The Bash harness includes a default-root regression check that validates output directory resolution when `--out-dir` is omitted.
- The `-Quick` flag filters execution to smoke-test cases marked with `"quick": true` for rapid validation cycles.

## Frequently Asked Questions

### Where are the regression test cases defined in reverse-skill?

All test cases are defined in [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json). Each entry specifies a user hint, an expected skill ID, and an optional quick-test flag. The JSON structure allows the PowerShell and Bash harnesses to load and iterate over scenarios without hardcoding test logic.

### How do I run only the quick smoke tests?

Pass the `-Quick` switch to the PowerShell harness: `powershell -File skills/scripts/test-routing.ps1 -Quick`. This filters the benchmark to only cases where the `quick` property is `true`, significantly reducing execution time for rapid validation during development.

### What is the default-root regression check?

The default-root regression check is a specific validation in [`skills/scripts/test-routing.sh`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/scripts/test-routing.sh) (lines 70-78) that ensures the router writes output to the caller’s project directory when no `--out-dir` argument is provided. This prevents the router from incorrectly placing files in its own installation directory.

### Which files parse the routing output during testing?

The PowerShell harness uses `skills/scripts/lib/RouteScope.ps1` to extract the `Id` field from [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md), while the Bash harness uses grep-based parsing to locate the `- primary:` skill identifier. Both methods verify that the router selected the expected skill defined in the benchmark.