How to Validate Routing Coherence with `test-routing.sh`

test-routing.sh is a Bash test harness that validates reverse-skill routing logic against a canonical benchmark and verifies default-root regression behavior.

In the zhaoxuya520/reverse-skill repository, routing coherence ensures that user hints correctly map to their intended skills. The test-routing.sh script automates this validation through two complementary test suites, catching regressions before they reach production.

---

What test-routing.sh Does

The script performs benchmark validation and default-root regression testing. These checks confirm both that individual routing cases work correctly and that the Bash router matches the behavior of its PowerShell counterpart when invoked without explicit directory arguments.

---

Benchmark Validation (Lines 41–66)

This first test suite iterates through every case defined in skills/tests/routing-benchmark.json. For each entry, the script:

1. Launches master-route.sh with the supplied hint
2. Captures the generated route-scope.md output
3. Extracts the primary: field (the routed skill)
4. Compares it against the expected skill in the benchmark

The loop runs from line 41 to line 66 of skills/scripts/test-routing.sh. Each execution result is recorded and tallied for the final report.

---

Default-Root Regression (Lines 70–128)

When invoked without --project-root or --out-dir, the Bash router must behave like the PowerShell implementation and write the scope file to the caller's project directory.

This test constructs a temporary fixture package, copies the router and configuration files, then executes from a separate caller project. It verifies:

• Exactly one route-scope.md exists under the caller's work/ folder
• The project_root field in the scope file matches the caller's directory

Any deviation causes a [FAIL] entry and non-zero exit status.

---

Running the Validation

Execute the script from the repository root:

bash skills/scripts/test-routing.sh

The script auto-detects the Python interpreter, loads the benchmark, executes all routing cases, and prints a concise pass/fail report.

Capturing Output

bash skills/scripts/test-routing.sh 2>&1 | tee routing-test.log

This pipes both stdout and stderr to a log file while preserving terminal visibility.

---

Interpreting Test Output

Output Pattern	Meaning
[PASS]	Successful routing case or default-root regression
[FAIL]	Mismatch: shows hint, expected skill, actual result, and exit code (e.g., hint='ssh' expect='ssh' got='ERR' exit=1)
TOTAL=xx PASS=xx FAIL=0	Final tally; PASS should equal total benchmark cases

A successful run concludes with:

OVERALL: ALL PASS (xx routing cases + default-root regression)

Any [FAIL] entry causes CI pipelines to flag the regression via non-zero exit status.

---

Debugging Failed Cases

Inspect a Specific Failure Manually

Suppose the hint "ftp" fails. Locate and examine the generated scope file:

cat work/master-route-*/route-scope.md | grep -A2 "ftp"

This reveals what the router actually produced versus the expected routing.

Verify Router Execution Directly

# Test a single hint

bash skills/scripts/master-route.sh --hint "git-commit" --project-root /tmp/test

Compare the output against skills/tests/routing-benchmark.json to identify discrepancies.

---

Adding New Routing Cases

1. Edit skills/tests/routing-benchmark.json and append a new object with "hint" and "expect" fields
2. Ensure the corresponding skill directory exists at skills/<skill-name>/
3. Re-run bash skills/scripts/test-routing.sh to verify the new case passes

Example benchmark entry:

{
  "hint": "docker-compose",
  "expect": "docker"
}

---

Key Files in the Routing System

• skills/scripts/test-routing.sh — Bash test harness driving benchmark validation and default-root regression
• skills/tests/routing-benchmark.json — JSON list of routing test cases (hint → expected primary skill)
• skills/scripts/master-route.sh — Core Bash router invoked by the test script
• skills/config/routing.json — Central routing configuration defining skill mappings

---

Summary

• test-routing.sh validates routing coherence through benchmark comparison and default-root regression
• Benchmark validation (lines 41–66) checks every hint-to-skill mapping in routing-benchmark.json
• Default-root regression (lines 70–128) ensures Bash/PowerShell behavioral parity when directories are unspecified
• Exit status is non-zero on any failure, enabling CI integration
• Regular execution catches routing regressions before deployment

---

Frequently Asked Questions

What does "routing coherence" mean in reverse-skill?

Routing coherence means the master-route.sh router consistently maps user hints to their canonical skills as defined in skills/config/routing.json. The test-routing.sh script enforces this by comparing actual router output against the expected results stored in skills/tests/routing-benchmark.json.

How do I run test-routing.sh in a CI pipeline?

Execute bash skills/scripts/test-routing.sh as a test step. The script exits with status 0 on full success and non-zero on any failure, so standard CI failure detection applies. Pipe output to capture logs: bash skills/scripts/test-routing.sh 2>&1 | tee -a build.log.

What should I do if default-root regression fails?

First, verify that master-route.sh handles missing --project-root and --out-dir arguments by defaulting to the caller's working directory. Check lines 70–128 of test-routing.sh to understand the fixture structure, then trace the router's path resolution logic in skills/scripts/master-route.sh.

Can I test a single routing case instead of the full suite?

The script does not support single-case execution natively. For targeted debugging, manually invoke master-route.sh with your hint and compare output against routing-benchmark.json. For permanent single-case testing, temporarily edit routing-benchmark.json to contain only your target case.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}