How test-routing.ps1 Validates PRIMARY Skill Mapping for Hints in PowerShell

test-routing.ps1 validates PRIMARY skill mapping by executing the router against a canonical benchmark of 163 hint-to-skill pairs, capturing the actual output, and asserting exact equality with the expected PRIMARY identifier.

This PowerShell regression harness is part of the zhaoxuya520/reverse-skill repository. It provides continuous integration protection for the routing engine, ensuring that changes to routing.json or master-route.ps1 do not silently alter which skill gets selected for a given user hint.

Loading the Routing Benchmark

The script begins by locating and loading the canonical test data.

At line 22, test-routing.ps1 constructs the path to skills/tests/routing-benchmark.json using Join-Path:

$benchmarkPath = Join-Path $skillsRoot 'tests\routing-benchmark.json'
$benchmark = Get-Content $benchmarkPath | ConvertFrom-Json

This JSON file contains 163 structured cases. Each case specifies:

• hint: The input string passed to the router
• primary: The expected PRIMARY skill identifier
• quick: A boolean flag for smoke-test filtering

Optional Quick Filter for CI Speed

When the -Quick switch is present, the script filters to a subset of approximately 40 cases marked with "quick": true.

Line 35 implements this conditional filter:

if ($Quick) {
    $benchmark = $benchmark | Where-Object { $_.quick -eq $true }
}

This allows CI pipelines to run a fast smoke test before executing the full regression suite.

Executing the Router and Capturing Output

For each case in the benchmark, the script extracts the hint and expected PRIMARY, then invokes master-route.ps1 in a separate PowerShell process.

Line 58 shows the core invocation:

$routerOutput = & powershell.exe -NoProfile -File $masterRoutePath -Hint $case.hint

The router prints a line formatted as PRIMARY -> skills/<path>. The script parses this output to extract $actualPrimary.

Assertion and Failure Collection

Line 71 performs the critical comparison:

if ($actualPrimary -ne $case.primary) {
    Write-Error "[FAIL] Hint: $($case.hint)`n       Expected: $($case.primary)`n       Got:      $actualPrimary"
    $fails += @{ Hint = $case.hint; Expected = $case.primary; Actual = $actualPrimary }
}

Mismatches are logged to the console and accumulated in the $fails array for the final summary.

Diagnostic Logging

When -LogDir <directory> is provided, line 65 writes per-case logs:

if ($LogDir) {
    $logPath = Join-Path $LogDir "$($case.id).log"
    $routerOutput | Set-Content $logPath
}

This captures full stdout/stderr from each router invocation, enabling post-hoc debugging of routing failures.

Exit Code and CI Integration

The script concludes with a pass/fail summary and appropriate exit code. Line 84 determines the final result:

Write-Host "[PASS] $passes/$total hints → expected PRIMARY matched"
if ($fails.Count -gt 0) {
    exit 1
}

A non-zero exit code causes the CI pipeline to fail, blocking merges that introduce routing regressions.

Complete Usage Examples

Run the full 163-case benchmark:

.\skills\scripts\test-routing.ps1

Execute only the quick smoke-test subset:

.\skills\scripts\test-routing.ps1 -Quick

Use a custom benchmark file with diagnostic logging:

.\skills\scripts\test-routing.ps1 `
    -Benchmark .\my-custom-benchmark.json `
    -LogDir .\router-logs

Validation Pipeline Architecture

The test-routing.ps1 script sits at the center of a three-layer validation system:

Component	Responsibility	Source Path
Benchmark data	Defines correct hint-to-PRIMARY mappings	skills/tests/routing-benchmark.json
Router implementation	Computes PRIMARY from hint via routing.json	skills/scripts/master-route.ps1
Regression harness	Asserts equality between actual and expected	skills/scripts/test-routing.ps1

Changes to any layer trigger the harness. If routing.json rules are modified, master-route.ps1 behavior shifts, and test-routing.ps1 surfaces any resulting PRIMARY mismatches immediately.

Key Source Files

• skills/scripts/test-routing.ps1 — The regression runner validated in this article
• skills/scripts/master-route.ps1 — Core routing engine under test
• skills/tests/routing-benchmark.json — Canonical 163-case test suite
• skills/config/routing.json — Routing rules defining PRIMARY selection logic
• skills/MASTER-ROUTING.md — Documentation of PRIMARY fast-path semantics

Summary

• test-routing.ps1 loads routing-benchmark.json as the source of truth for expected PRIMARY mappings
• The -Quick switch enables fast CI smoke tests on ~40 critical cases
• Each case executes master-route.ps1 -Hint "<hint>" and parses the PRIMARY -> output line
• Actual and expected PRIMARY values are compared with strict string equality
• Failures are collected, logged, and cause exit code 1 to fail the CI pipeline
• The -LogDir parameter enables per-case stdout/stderr capture for debugging

Frequently Asked Questions

What format must the benchmark JSON follow?

Each case requires three fields: hint (string, the input to route), primary (string, the expected skill path), and quick (boolean, optional filter flag). The primary value must match the exact string output by master-route.ps1 after the PRIMARY -> prefix.

How does the script isolate router execution from the test harness?

It spawns a fresh powershell.exe process for each hint rather than dot-sourcing or invoking the router in the same session. This prevents routing code from accidentally inheriting test variables or modified state, ensuring deterministic validation.

Can I run the harness against a modified routing configuration?

Yes. Since master-route.ps1 reads skills/config/routing.json at runtime, any changes to that file are automatically picked up during validation. The benchmark file path is also configurable via the -Benchmark parameter for testing experimental mappings.

Why does the script use Write-Error for failures instead of throwing exceptions?

Write-Error logs to the error stream without halting iteration, allowing the harness to collect all mismatches in a single run. This provides complete visibility into routing regressions rather than stopping at the first failure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}