How test-routing.ps1 Validates 173 Benchmark Cases in the reverse-skill Repository

The test-routing.ps1 regression suite validates 173 benchmark cases by loading the JSON benchmark definition, executing the master-route.ps1 logic against each hint, parsing the resulting route-scope.md output, and asserting that the extracted route identifier matches the expected value.

The reverse-skill repository relies on a deterministic PowerShell regression framework to guarantee routing consistency across changes. Located at skills/scripts/test-routing.ps1, this automated suite serves as the primary validation gate, ensuring that all 173 routing hints defined in the benchmark dataset resolve to their designated route identifiers.

Benchmark Data Structure and Case Selection

The suite reads its test matrix from skills/tests/routing-benchmark.json, which contains 173 discrete routing scenarios. Each entry follows a consistent schema with a hint (the user query), an expect field (the target route identifier), and an optional quick boolean flag for smoke testing.

Loading and Filtering Cases

The script deserializes the benchmark into a PowerShell object using ConvertFrom-Json. By default, the runner executes every case. When the -Quick switch is provided, the suite filters to only those entries marked with "quick": true, enabling rapid validation of critical paths during development.

$bm = Get-Content -LiteralPath $Benchmark -Raw -Encoding UTF8 | ConvertFrom-Json
$cases = @($bm.cases)

if ($Quick) {
    $cases = @($cases | Where-Object { $_.quick })
}

Test Execution Pipeline

For each selected case, the suite performs an isolated validation loop that exercises the full routing stack from hint ingestion to identifier extraction.

Invoking the Routing Entry Point

The script invokes skills/scripts/master-route.ps1 (the primary routing logic) through the reverse-skill host executable stored in $HostExe. It passes the hint via the -Hint parameter and isolates output to a unique temporary directory specified by -OutDir.

$tmp = Join-Path $tmpBase ("rs-rt-{0}" -f [guid]::NewGuid().ToString('n'))

try {
    $null = & $HostExe -NoProfile -ExecutionPolicy Bypass `
             -File $masterRoute -Hint $c.hint -OutDir $tmp 2>&1
    # Parsing logic follows...

} catch {
    $got = 'EXC:' + $_.Exception.Message
}

Parsing route-scope.md with Get-ReverseRouteScopeFields

After execution, the suite locates the generated route-scope.md file within the temporary directory. It utilizes the Get-ReverseRouteScopeFields helper function defined in skills/scripts/lib/RouteScope.ps1 to parse the markdown content and extract the routed identifier into $parsed.Id.

$scope = Join-Path $tmp 'route-scope.md'

if (Test-Path $scope) {
    $text = Get-Content $scope -Raw -Encoding UTF8
    $parsed = Get-ReverseRouteScopeFields -Text $text
    if ($parsed.Id) { $got = $parsed.Id }
}

Assertion and Cleanup

The extracted identifier ($got) is compared against the benchmark expectation ($c.expect). Matching values increment the pass counter, while mismatches are recorded in the failure collection. The finally block ensures temporary directories are removed after each iteration to prevent cross-test contamination.

if ($got -eq $c.expect) {
    $pass++
    $detail.Add("PASS $($c.expect) -> $($c.hint)")
} else {
    $failCount++
    $msg = "FAIL hint='$($c.hint)' expect=$($c.expect) got=$got"
    $fail.Add($msg)
    $detail.Add($msg)
}
finally {
    Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue
}

Reporting and Exit Codes

Upon completion of all 173 cases, the suite generates three artifacts in the output directory:

• SUMMARY.txt: Contains aggregate statistics including total cases executed, passed count, failed count, and quick-mode status.
• cases.txt: A complete log of every test case result.
• failures.txt: Detailed records of only the failed assertions for debugging.

The script exits with code 0 when all cases pass, and code 1 if any failure is detected, enabling CI/CD integration that blocks merges on regression.

Summary

• The test-routing.ps1 script in skills/scripts/ serves as the automated regression harness for the reverse-skill repository.
• It validates 173 benchmark cases defined in skills/tests/routing-benchmark.json by comparing expected route identifiers against actual output from master-route.ps1.
• The Get-ReverseRouteScopeFields function in skills/scripts/lib/RouteScope.ps1 extracts the routed ID from generated route-scope.md files.
• Execution supports a -Quick mode for rapid validation of smoke-test subsets marked in the benchmark.
• Exit code 0 indicates total success, while 1 signals regression failures requiring investigation.

Frequently Asked Questions

How does the suite handle temporary files during execution?

Each benchmark case executes in an isolated temporary directory generated with a GUID-based name. After the routing logic completes and the route-scope.md file is parsed, the Remove-Item -Recurse -Force command deletes the temporary directory, ensuring no residual files persist between cases or contaminate subsequent test runs.

What is the purpose of the -Quick switch in test-routing.ps1?

The -Quick switch filters the 173-case benchmark dataset to only those entries marked with "quick": true in the JSON source. This allows developers to run a minimal smoke test set during iterative development without executing the full regression matrix, significantly reducing validation time before committing changes.

How does the suite determine if a routing test passed or failed?

The suite compares the route identifier extracted from route-scope.md (via Get-ReverseRouteScopeFields) against the expect field defined in the benchmark JSON. If the values match exactly, the case passes; if they differ, or if the routing script throws an exception captured in the try-catch block, the case is marked as a failure and logged to failures.txt.

Which file contains the actual routing logic being tested?

The primary routing entry point is master-route.ps1 located in skills/scripts/. This script receives the hint via the -Hint parameter, processes the routing decision, and writes the resulting scope metadata to route-scope.md within the specified output directory, which the test harness then validates against the benchmark expectations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}