How to Run the 163 Routing Benchmark Locally to Validate Routing Changes

Run the skills/scripts/test-routing.sh (Linux/macOS) or skills/scripts/test-routing.ps1 (Windows) script after modifying routing configuration to validate that all 163 routing cases still resolve to the expected primary skill.

The zhaoxuya520/reverse-skill repository ships with a regression test suite containing 163 routing benchmark cases defined in skills/tests/routing-benchmark.json. Running this benchmark locally ensures that changes to skills/config/routing.json or new skill additions do not break existing routing behavior.

Platform-Specific Runners

The repository provides two scripts to execute the benchmark, depending on your operating system.

Linux/macOS: Bash Runner with Python Helper

The skills/scripts/test-routing.sh script implements the benchmark runner for Unix-like systems. It requires Python 3 (python3 or python on $PATH).

The script performs these operations (see lines 21–35 and 41–66 in the source):

1. Reads skills/tests/routing-benchmark.json
2. Invokes master-route.sh for each test case
3. Extracts the primary skill from the generated route-scope.md
4. Compares against the expect field in the benchmark

Windows: Native PowerShell Runner

The skills/scripts/test-routing.ps1 script provides equivalent functionality without external dependencies—only PowerShell is required.

Step-by-Step Workflow

1. Prepare your environment

   Verify Python 3 availability for Linux/macOS:

   python3 --version

   Windows users need no additional setup beyond PowerShell.

2. Apply your routing changes

   Edit skills/config/routing.json or add new skill definitions as needed.

3. Execute the benchmark

   Linux/macOS:

   cd skills/scripts
   ./test-routing.sh

   Windows:

   cd skills\scripts
   .\test-routing.ps1

4. Interpret results

   A successful run displays:

   
   TOTAL=203 PASS=203 FAIL=0
   

   Failures list the hint, expected skill, and actual primary result.

5. Iterate until green

   Adjust routing.json or skill implementations until the 163 routing benchmark passes completely.

Default-Root Regression Testing

The benchmark includes a specialized default-root regression that validates router behavior when invoked without --project-root or --out-dir flags. This ensures route-scope.md is written to the caller's project directory—a behavior mirrored between the Bash and PowerShell implementations.

Key Files for Reference

File	Purpose
skills/tests/routing-benchmark.json	163 test cases mapping hints to expected primary skills
skills/config/routing.json	Central routing table consulted by the router
skills/scripts/test-routing.sh	Bash benchmark runner with embedded Python helper
skills/scripts/test-routing.ps1	Native PowerShell benchmark runner
skills/scripts/verify-routing-coherence.ps1	CI verification ensuring routing and benchmark synchronization
skills/scripts/master-route.sh	Bash router implementation (invoked by benchmark)
skills/scripts/master-route.ps1	PowerShell router implementation (invoked by benchmark)

Summary

• Run the 163 routing benchmark via test-routing.sh or test-routing.ps1 immediately after any routing modification
• The benchmark validates that primary skill selection matches expectations across all 163 cases
• Exit status is non-zero on failure, causing CI pipelines to break
• Both runners extract results from route-scope.md and compare against routing-benchmark.json

Frequently Asked Questions

What is the 163 routing benchmark?

The 163 routing benchmark is a regression test suite in skills/tests/routing-benchmark.json containing 203 total test cases—including 163 core routing cases plus additional regression tests. Each case specifies a hint and the expected primary skill that the router should select.

Do I need to install Python on Windows to run the benchmark?

No. The skills/scripts/test-routing.ps1 runner uses native PowerShell with no external dependencies. Only the Linux/macOS test-routing.sh script requires Python 3.

What happens if the benchmark detects a routing mismatch?

The runner prints FAIL details including the hint, expected skill, and actual result, then exits with a non-zero status. This causes CI builds to fail and prevents broken routing changes from merging.

How does the benchmark verify correct routing?

The Python helper embedded in test-routing.sh (and equivalent PowerShell logic) calls the router for each case, parses the generated route-scope.md file for the primary entry, and compares it against the expect field in the benchmark JSON.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}