How 163 Routing Benchmark Cases Ensure Comprehensive Coverage Across All Routing Rules

The 163 routing benchmark cases in reverse-skill enforce a strict one-to-one mapping between every routing rule and at least one test case, supported by bilingual hints and automated coherence verification.

The zhaoxuya520/reverse-skill project relies on a single benchmark file to validate its entire routing matrix. The 163 routing benchmark cases stored in skills/tests/routing-benchmark.json act as the definitive contract for skills/config/routing.json, ensuring that each rule is triggered correctly by realistic user queries.

Verified One-to-One Rule Coverage

Every routing rule defined in skills/config/routing.json—from R1 (APK reverse engineering) through R25 (Windows/AD) and later additions—must appear in the benchmark. The PowerShell script skills/scripts/verify-routing-coherence.ps1 iterates over the benchmark's cases array and confirms that each rule identifier has at least one matching expect value.

This mechanism prevents orphaned rules. If a rule exists in routing.json but lacks a corresponding case in routing-benchmark.json, the verification script fails and blocks the CI pipeline.

Bilingual Hint Coverage

The benchmark does not test rules with monolingual queries alone. For the majority of rules, the 163 routing benchmark cases provide paired entries: one English hint and one Chinese hint that both resolve to the same rule identifier.

For example, the benchmark contains an English case such as decompile APK with jadx apktool smali and a Chinese case such as 安卓 APK 加固 反编译, both expecting R1. This design proves that the routing regexes are robust across language boundaries without maintaining separate test suites.

Tiered Validation Strategy

The benchmark supports two execution modes to balance speed against thoroughness.

Quick-Smoke Subset for Fast Feedback

Cases marked with "quick": true form a minimal regression set of approximately 40 cases. Running skills/scripts/test-routing.ps1 with the -Quick flag executes only this subset, giving developers immediate validation that every rule still resolves correctly after small changes. The benchmark meta field "quickNote" explicitly defines these as the minimal regression set per route.

Full Regression in CI

The complete suite—all 163 bilingual cases producing roughly 200 total hint validations—is executed by skills/scripts/test-routing.ps1 without flags. The CI workflow defined in .github/workflows/ci.yml runs this full regression on every push, ensuring that no change to routing.json breaks existing routing behavior.

Self-Documenting Test-Driven Routing

The benchmark serves as more than a test suite; it drives the documentation workflow. When contributors discover a new routing scenario, the prescribed process—codified in RULES.md and CONTRIBUTING.md—is to add a failing case to skills/tests/routing-benchmark.json, update skills/config/routing.json until the test passes, and then merge.

This tight coupling means the 163 routing benchmark cases function as executable documentation for the routing matrix. The file paths themselves tell the story: skills/tests/routing-benchmark.json validates skills/config/routing.json, checked by verify-routing-coherence.ps1, and enforced by .github/workflows/ci.yml.

Running the Benchmark

You can execute the validation scripts directly from the repository root.

Run the full regression suite:

powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1

Run only the quick-smoke cases:

powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1 -Quick

To add coverage for a new rule, append a case object to the cases array in skills/tests/routing-benchmark.json:

{
    "hint": "your new query here",
    "expect": "R12",
    "quick": false
}

After adding the case, run skills/scripts/test-routing.ps1 to confirm that the hint resolves to the expected rule identifier.

Summary

• The skills/scripts/verify-routing-coherence.ps1 script enforces a strict one-to-one mapping between every rule in routing.json and at least one case in the 163 routing benchmark cases.
• Bilingual English and Chinese hints ensure routing regexes work across languages without duplicate suites.
• A quick subset of approximately 40 cases enables fast local feedback, while the full suite runs in CI via .github/workflows/ci.yml.
• The benchmark acts as executable documentation, with RULES.md and CONTRIBUTING.md codifying the test-driven workflow for adding new routes.

Frequently Asked Questions

How does the benchmark guarantee that no routing rule is left untested?

The skills/scripts/verify-routing-coherence.ps1 script parses both skills/config/routing.json and skills/tests/routing-benchmark.json. It asserts that every rule identifier appears as an expect value in at least one benchmark case. If any rule lacks a matching case, the script exits with an error and fails the CI build.

What is the difference between the quick and full benchmark runs?

The quick run invokes skills/scripts/test-routing.ps1 with the -Quick flag and evaluates only cases where "quick": true—roughly 40 cases that form a minimal regression set. The full run evaluates all 163 bilingual cases, approximately 200 total hints, and is executed automatically by the CI pipeline on every push.

Why does the benchmark include both English and Chinese hints?

Each major routing rule is represented by paired English and Chinese test cases that share the same expect rule identifier. This bilingual coverage validates that the regex-based routing logic in routing.json correctly handles queries across both languages without requiring language-specific rule duplication.

How do I add a new case to the 163 routing benchmark?

Add a JSON object containing hint, expect, and quick fields to the cases array inside skills/tests/routing-benchmark.json. Run skills/scripts/test-routing.ps1 to confirm the new hint resolves to the intended rule. If you also added a new rule to routing.json, verify that skills/scripts/verify-routing-coherence.ps1 still passes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}