# How to Run Routing Regression Tests After Modifying routing.json

> Run routing regression tests after modifying routing.json. Execute test-routing.ps1 to ensure task-to-skill mappings remain accurate against benchmark data. Validate your routing logic now.

- Repository: [ZhaoXu/reverse-skill](https://github.com/zhaoxuya520/reverse-skill)
- Tags: how-to-guide
- Published: 2026-08-18

---

**Execute `skills/scripts/test-routing.ps1` to validate that all task-to-skill mappings in [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) still route correctly against the benchmark dataset.**

The `zhaoxuya520/reverse-skill` repository centralizes routing logic in [`skills/config/routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/config/routing.json). Any modification to this file risks breaking existing task classifications, so automated regression testing is essential. The `test-routing.ps1` script provides comprehensive verification by replaying benchmark cases through the router and flagging discrepancies.

## What the Routing Regression Test Validates

The regression suite confirms three critical properties of your [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) changes:

- **Keyword rule accuracy** — Every hint in the benchmark resolves to the expected primary route
- **Priority ordering correctness** — The `priority` list in each route definition is respected during scoring
- **Zero regression introduction** — Previously working cases continue to produce identical results

The benchmark itself lives in [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json), containing approximately 162 hint-to-route pairs that exercise the full surface area of the routing logic.

## How test-routing.ps1 Works

### Benchmark Loading and Filtering

At lines 21–24, the script initializes the default benchmark path:

```powershell

# From skills/scripts/test-routing.ps1 L21-L24

$Benchmark = Join-Path $PSScriptRoot ".." "tests" "routing-benchmark.json" | Resolve-Path
$benchmark = Get-Content $Benchmark | ConvertFrom-Json

```

The `-Quick` switch (lines 40–41) collapses the benchmark to one representative case per route, enabling rapid iteration during development.

### Router Invocation and Result Extraction

For each benchmark case, the script:

1. Calls `master-route.ps1` with the hint
2. Parses the generated [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md)
3. Extracts the `primary` route identifier (lines 55–58)

```powershell

# Core verification logic from skills/scripts/test-routing.ps1 L55-L58

$routeScope = Get-Content "$WorkDir\route-scope.md" -Raw
if ($routeScope -match 'primary:\s*(\S+)') {
    $actualPrimary = $matches[1]
}

```

The actual primary is compared against the expected value from the benchmark. Mismatches are logged with full context for debugging.

### Exit Codes and Summary Generation

Lines 86–94 handle result aggregation and final status:

- Exit code **0** — All cases passed
- Exit code **1** — One or more cases failed
- [`SUMMARY.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/SUMMARY.txt) — High-level statistics (total, pass, fail)
- [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt) — Detailed mismatch records (lines 90–92)

## Running the Regression Tests

### Full Regression Suite

Execute all 162 benchmark cases to ensure comprehensive coverage:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1

```

### Quick Regression for Iteration

Use the `-Quick` flag during active development to reduce feedback time:

```powershell
powershell -File skills/scripts/test-routing.ps1 -Quick

```

### Custom Benchmark or Log Locations

Point to alternative test datasets or controlled output directories:

```powershell
powershell -File skills/scripts/test-routing.ps1 `
    -Benchmark .\my-custom-benchmark.json `
    -LogDir C:\Users\Me\reverse-skill-logs

```

## Interpreting Test Output

A typical execution produces console output followed by detailed logs:

```

=== test-routing | 162 cases (quick=False) ===
PASS R1 -> decompile APK with jadx apktool smali
...
[FAIL] hint='playwright browser automation' expect=R19 got=R3
...
=== ROUTING TEST SUMMARY ===
TOTAL=162
PASS=158
FAIL=4
QUICK=False
LogDir=C:\Users\Me\AppData\Local\Temp\rs-routing-test-20260809-142530
OVERALL: FAIL (4)

```

When failures occur, inspect [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt) in the reported log directory to identify which keyword patterns need adjustment in [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json).

## How master-route.ps1 Consumes routing.json

The regression test exercises `skills/scripts/master-route.ps1`, which implements the actual routing engine. This script:

- Loads [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) at lines 22–30
- Evaluates keyword matches against each route's `patterns` array
- Scores candidates and selects the primary based on the `priority` list (corresponding to lines 108–110 in the JSON structure)

Your modifications to [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) directly affect this scoring pipeline, making the regression test the definitive validation mechanism.

## Summary

- **Invoke `test-routing.ps1`** after any [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) edit to prevent regressions
- **Use `-Quick`** for rapid feedback during iterative development
- **Check [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt)** in the log directory to debug mismatches
- **Expect exit code 0** only when all benchmark cases pass
- **Coordinate changes** between [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) and [`routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing-benchmark.json) when adding new routes

## Frequently Asked Questions

### What exit code does test-routing.ps1 return on failure?

The script exits with code **1** when any benchmark case fails, and **0** when all pass. This enables integration with CI/CD pipelines that depend on explicit status codes.

### Where are regression test logs stored by default?

Logs are written to a timestamped subdirectory under `%TEMP%` with the pattern `rs-routing-test-<yyyyMMdd-HHmmss>`. Use `-LogDir` to override this location.

### Can I add new test cases to the benchmark?

Yes. Append objects to [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json) with `hint` and `expected_route` properties. The script loads this file dynamically, so no code changes are required.

### How does the quick mode select representative cases?

When `-Quick` is specified, the script filters to one case per unique expected route, ensuring each route is exercised at least once without running the full 162-case suite.