# How to Run the Routing Regression Tests for reverse-skill

> Easily run reverse-skill routing regression tests. Execute the full suite with skills/scripts/test-ps1 or use -Quick for minimal case testing.

- Repository: [ZhaoXu/reverse-skill](https://github.com/zhaoxuya520/reverse-skill)
- Tags: how-to-guide
- Published: 2026-08-10

---

**Run `skills/scripts/test-routing.ps1` with PowerShell to execute the full routing regression suite, or add `-Quick` to test only the minimal subset of cases.**

The routing regression tests in the **zhaoxuya520/reverse-skill** repository ensure that every `hint → PRIMARY` mapping defined in the benchmark file still resolves correctly after any change to the routing rules. This guide covers the complete workflow from prerequisites through interpreting test results.

## What the Routing Regression Tests Verify

The regression suite validates that `master-route.ps1` consistently returns the expected **PRIMARY** skill identifier for every registered routing hint. Each test case in [`routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing-benchmark.json) contains:

- A **hint** — the user input pattern
- An **expected PRIMARY** — the skill route that should be selected
- A **quick flag** — whether this case is included in fast validation runs

When you modify [`skills/config/routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/config/routing.json), these tests confirm you haven't broken existing mappings.

## Prerequisites

Before running the tests, ensure your environment meets these requirements:

- **PowerShell 5.1+** (Windows) or **PowerShell Core** (Linux/macOS)
- The repository's **tool index** must be refreshed so external dependencies are detected
- Access to `%TEMP%` on Windows or `/tmp` on Unix systems for log output

## Step 1: Refresh the Tool Index

The reverse-skill repository maintains an indexed registry of available tools. Run this once after cloning:

```powershell

# Windows

powershell -File skills/scripts/refresh-tool-index.ps1

# Linux / macOS

bash skills/scripts/refresh-tool-index.sh

```

This step is required before `master-route.ps1` can correctly resolve routing decisions.

## Step 2: Execute the Routing Regression Test

The primary test runner is **`skills/scripts/test-routing.ps1`**. It handles benchmark loading, temporary workspace creation, per-case execution, and result aggregation.

### Run the Full Benchmark (Default, 163 Cases)

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1

```

This validates every hint-to-route mapping in the benchmark file. Execution time depends on case complexity.

### Run Quick Mode (Minimal Subset)

```powershell
powershell -File skills/scripts/test-routing.ps1 -Quick

```

**Quick mode** executes only cases marked `"quick": true` in [`routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing-benchmark.json) — approximately one case per route. Use this for rapid feedback during development.

### Custom Benchmark and Log Directory

```powershell
powershell -File skills/scripts/test-routing.ps1 `
    -Benchmark path\to\my-benchmark.json `
    -LogDir C:\temp\my-routing-log

```

The `-Benchmark` parameter overrides the default location. The `-LogDir` parameter controls where test artifacts are written (defaults to a timestamped subdirectory of `%TEMP%` or `/tmp`).

## Understanding Test Output

A successful run produces output similar to:

```

=== test-routing | 163 cases (quick=False) ===
PASS R1 -> decompile APK with jadx apktool smali
...
=== ROUTING TEST SUMMARY ===
TOTAL=163
PASS=163
FAIL=0
QUICK=False
LogDir=C:\Users\<name>\AppData\Local\Temp\rs-routing-test-20260810-152300
OVERALL: ALL PASS (163)

```

### Log Directory Contents

Each run creates a unique directory containing:

| File | Purpose |
|------|---------|
| [`cases.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/cases.txt) | Per-case execution details |
| [`SUMMARY.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/SUMMARY.txt) | Aggregated statistics |
| [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt) | List of failed cases (only if `FAIL > 0`) |

### Interpreting Failures

If any case fails, [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt) contains lines like:

```

FAIL hint='disassemble ARM binary' expect=R7 got=EXC:tool_not_found

```

The exit code is `1` on failure. Use `$LASTEXITCODE` in PowerShell to detect this programmatically:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1
if ($LASTEXITCODE -eq 0) {
    Write-Host "All routing cases passed."
} else {
    Write-Host "Some routing cases failed – inspect $((Get-ChildItem $env:TEMP -Filter 'rs-routing-test-*' | Sort-Object LastWriteTime -Descending | Select-Object -First 1).FullName)"
}

```

## Key Files in the Routing Test Architecture

| File | Role |
|------|------|
| `skills/scripts/test-routing.ps1` | Test orchestrator — loads benchmark, invokes `master-route.ps1`, aggregates results (lines 21–96) |
| [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json) | JSON array of test cases with hint, expected PRIMARY, and quick flag (lines 9–13+) |
| `skills/scripts/master-route.ps1` | Core routing logic that reads [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json) and produces [`route-scope.md`](https://github.com/zhaoxuya520/reverse-skill/blob/main/route-scope.md) |
| [`skills/config/routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/config/routing.json) | Canonical routing rules (R0–R40); any modification requires regression verification |

## Automated Testing in CI Pipelines

Integrate the routing regression tests into your build process:

```powershell

# Run quick validation on every commit

powershell -File skills/scripts/test-routing.ps1 -Quick
if ($LASTEXITCODE -ne 0) { throw "Routing regression failed" }

# Run full suite before releases

powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1
if ($LASTEXITCODE -ne 0) { throw "Full routing regression failed" }

```

The `-NoProfile` and `-ExecutionPolicy Bypass` flags ensure consistent execution in restricted environments.

## Summary

- **Primary test runner**: `skills/scripts/test-routing.ps1`
- **Benchmark source**: [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json) — the single source of truth for regression cases
- **Quick mode**: Use `-Quick` for rapid validation of one case per route
- **Success indicator**: Exit code `0` with `PASS` equal to `TOTAL`
- **Failure diagnosis**: Check [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt) in the generated log directory for detailed mismatch information
- **Prerequisite step**: Refresh tool index with `refresh-tool-index.ps1` or `.sh` before first run

## Frequently Asked Questions

### What PowerShell version is required for the routing regression tests?

PowerShell 5.1 or later on Windows, or PowerShell Core on Linux and macOS. The script uses modern parameter binding and temp file handling that requires these versions.

### Where does the routing regression test get its expected results?

From [`skills/tests/routing-benchmark.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/skills/tests/routing-benchmark.json), a JSON file containing 163 test cases as of the current repository state. Each case specifies an input hint, the expected PRIMARY route identifier, and whether it belongs to the quick test subset.

### Can I run the routing regression tests without executing the full 163 cases?

Yes. Add the `-Quick` switch to `test-routing.ps1` to run only cases where `"quick": true` in the benchmark file — typically one representative case per routing rule rather than exhaustive coverage.

### What should I do if a routing test fails after modifying [`routing.json`](https://github.com/zhaoxuya520/reverse-skill/blob/main/routing.json)?

Examine [`failures.txt`](https://github.com/zhaoxuya520/reverse-skill/blob/main/failures.txt) in the generated log directory to identify which hint produced an unexpected PRIMARY. The file shows expected versus actual results, including exception details if routing failed entirely. Revert your change or update the benchmark if the new behavior is intentional.