How to Add Regression Test Cases for New Routing Rules in test-routing.ps1

To add a regression test for a new routing rule in the reverse-skill repository, append a [PSCustomObject]@{ Hint = 'user hint'; Expect = 'skills/path/file.md' } entry to the $TestCases array inside skills/scripts/test-routing.ps1 and verify the match locally before committing.

The reverse-skill project relies on a JSON-driven routing subsystem to translate natural language hints into executable skill files. When you modify routing logic or add new intent mappings, you must validate those changes through test-routing.ps1 to prevent regressions. This script maintains a suite of 162+ test cases that assert every expected hint resolves to the correct skill path through the master-route.ps1 engine.

Architecture of the Routing Test System

The regression harness operates across three core files that define, execute, and validate routing behavior.

Component	Purpose	Source Path
routing.json	Central routing table mapping target types and intents to tool chains.	skills/config/routing.json
master-route.ps1	Entry point resolver that processes hints and returns concrete skill paths.	skills/scripts/master-route.ps1
test-routing.ps1	Automated regression suite that validates every rule against expected outcomes.	skills/scripts/test-routing.ps1

How the Test Harness Works

The test-routing.ps1 script executes a four-phase validation sequence:

1. Load Configuration: Reads the routing table from skills/config/routing.json.
2. Iterate Cases: Loops through the static $TestCases array defined within the script.
3. Invoke Router: Calls master-route.ps1 for each Hint, comparing the returned path against the Expect property.
4. Fail Fast: Exits with a non-zero code if any assertion fails, breaking CI pipelines immediately.

Each entry in the $TestCases array is a [PSCustomObject] containing:

• Hint – The textual string a user would type to trigger the route.
• Expect – The expected skill file path (e.g., skills/category/tool/file.md).
• Auth (optional) – Boolean flag indicating the test requires a granted authentication status.
• Env (optional) – Hashtable of environment variables required for the test context.

Step-by-Step Guide to Adding a Regression Test

Identify the Hint and Expected Skill

Before writing the test, determine the exact input string and its expected resolution.

Run the router manually to confirm the target path:

powershell -NoProfile -File skills/scripts/master-route.ps1 -Hint "scan a host"

Note the absolute path returned (e.g., skills/pentest-tools/nmap/scan-host.md). This value becomes your Expect property.

Locate the Test Cases Array

Open skills/scripts/test-routing.ps1 and scroll to the section declaring the $TestCases variable. The structure resembles:

$TestCases = @(
    [PSCustomObject]@{ Hint = "list tools"; Expect = "skills/tool-index.md" }
    # … 161 additional cases …

)

Append the New Test Object

Add a new [PSCustomObject] following the established pattern. Use single-quoted strings to prevent PowerShell variable expansion:

[PSCustomObject]@{
    Hint   = 'scan a host'
    Expect = 'skills/pentest-tools/nmap/scan-host.md'
}

For rules requiring authentication, include the Auth flag:

[PSCustomObject]@{
    Hint   = 'exfiltrate data'
    Expect = 'skills/data-exfiltration/exfil.md'
    Auth   = $true
}

Validate the Change Locally

Execute the regression suite to confirm your new case passes:

powershell -NoProfile -ExecutionPolicy Bypass -File skills/scripts/test-routing.ps1

The script outputs results for each case:

[PASS] Hint: scan a host → Expected: skills/pentest-tools/nmap/scan-host.md

Failures display the mismatched actual versus expected path, allowing immediate debugging.

Commit the Update

Add and commit the modified test script:

git add skills/scripts/test-routing.ps1
git commit -m "Add regression test for new 'scan a host' routing rule"

The repository's smoke and routing-regression pipelines automatically execute test-routing.ps1, ensuring the rule remains valid across future changes.

Practical Example: Testing a Decrypt File Rule

Consider adding a regression test for a new cryptographic routing rule. Insert the following object into the $TestCases array:

# In skills/scripts/test-routing.ps1

$TestCases = @(
    # Existing entries …

    # New regression case for OpenSSL decryption

    [PSCustomObject]@{
        Hint   = 'decrypt a file with openssl'
        Expect = 'skills/crypto/decrypt/openssl-decrypt.md'
    }
)

Running the test harness should produce:

[PASS] Hint: decrypt a file with openssl → Expected: skills/crypto/decrypt/openssl-decrypt.md

If the router returns skills/crypto/decrypt/gpg-decrypt.md instead, the test fails with a descriptive mismatch error, prompting immediate correction of either the routing rule or the test expectation.

Related Validation Scripts

Several auxiliary scripts support the routing test ecosystem:

• skills/scripts/verify-routing-coherence.ps1 – Validates JSON schema consistency in routing.json without executing skill paths.
• skills/scripts/smoke.ps1 – High-level integration test that bundles routing validation with other quick health checks for CI gates.

Summary

• Append new [PSCustomObject] entries to the $TestCases array in skills/scripts/test-routing.ps1 to protect routing changes.
• Include Hint and Expect properties for every case; add Auth = $true for authenticated routes.
• Execute test-routing.ps1 locally to catch mismatches before committing.
• Commit the updated script; CI automatically enforces the regression suite on pull requests.

Frequently Asked Questions

What structure must a test case follow in the $TestCases array?

Each test case must be a [PSCustomObject] with at minimum Hint and Expect string properties. The Hint represents the user input, while Expect contains the relative path to the skill file. Optional properties include Auth (boolean) and Env (hashtable) for scenarios requiring specific security contexts or environment variables.

How do I handle regression tests for authenticated routing rules?

Add the Auth property set to $true in your test case object. When present, the test harness ensures the authentication status is set to "granted" before invoking master-route.ps1, validating that privileged routing branches resolve correctly without exposing credentials in the test file itself.

Why does test-routing.ps1 exit with a non-zero code on failure?

The script exits with code 1 (or other non-zero values) when any hint fails to resolve to its expected path. This design integrates with CI/CD pipelines, causing automated builds to break immediately when a developer accidentally modifies routing logic that breaks existing user workflows.

Can I test environment-specific routing configurations?

Yes. Include an Env hashtable in your [PSCustomObject] definition to inject specific environment variables during the test execution. This allows regression testing of routes that behave differently based on system flags or deployment-specific settings without polluting the global environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}