How Cross-Platform CI Validates Routing Consistency on Windows and Ubuntu

The reverse-skill repository uses a GitHub Actions matrix strategy to execute identical PowerShell tests on both Windows and Ubuntu runners, ensuring routing logic behaves identically across platforms by validating 163 benchmark cases and verifying manifest coherence.

The zhaoxuya520/reverse-skill project maintains complex routing logic that must behave deterministically regardless of the underlying operating system. To guarantee cross-platform CI validation of routing consistency, the repository employs a sophisticated continuous integration pipeline that runs the same test suite against windows-latest and ubuntu-latest environments. This approach surfaces platform-specific regressions immediately by comparing actual routing outcomes against a canonical benchmark.

Matrix Strategy for Cross-Platform Validation

The CI workflow defines a job named routing-tests that uses a build matrix to parallelize execution across operating systems. This ensures that every code change is vetted in both Windows PowerShell and Linux pwsh environments.

Defining the OS Matrix

In .github/workflows/ci.yml (lines 11‑17), the strategy block declares a matrix including both target platforms:

jobs:
  routing-tests:
    strategy:
      matrix:
        os: [windows-latest, ubuntu-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v7

This configuration spawns two concurrent jobs—one on Windows and one on Ubuntu—both executing identical subsequent steps. Any divergence in behavior between the two runners causes the respective job to fail, alerting maintainers to platform-specific inconsistencies.

PowerShell Shim for Linux Compatibility

Because the Ubuntu runner ships with pwsh (PowerShell Core) while the test scripts invoke powershell, the workflow creates a symlink to ensure command uniformity. In .github/workflows/ci.yml (lines 21‑26), a conditional step runs only on Linux:

      - name: powershell shim (linux)
        if: runner.os == 'Linux'
        shell: bash
        run: sudo ln -sf "$(command -v pwsh)" /usr/local/bin/powershell

This shim guarantees that when test-routing.ps1 calls powershell, it resolves to the installed pwsh binary on Ubuntu, eliminating shell compatibility issues while keeping the test code platform-agnostic.

Routing Regression Testing (163 Cases)

The core validation logic resides in skills/scripts/test-routing.ps1, which performs a comprehensive regression against a canonical benchmark. The script loads skills/tests/routing-benchmark.json containing 163 test cases, executes the master routing script for each scenario, and validates the primary result.

Benchmark Execution Flow

According to the source at lines 30‑70, the script performs the following operations:

1. Loads the JSON benchmark into memory via ConvertFrom-Json
2. Iterates through every case in the $bm.cases array
3. Invokes master-route.ps1 with the specific hint and a temporary output directory
4. Parses the generated route-scope.md to extract the primary routing decision
5. Compares the actual result against $c.expect, recording any mismatch

# skills/scripts/test-routing.ps1 – core regression loop

$bm = Get-Content $Benchmark -Raw -Encoding UTF8 | ConvertFrom-Json
$cases = @($bm.cases)                     # ← load all 163 cases

foreach ($c in $cases) {
    $tmp = Join-Path $tmpBase ("rs-rt-{0}" -f [guid]::NewGuid().ToString('n'))
    $got = 'ERR'
    try {
        & powershell -NoProfile -ExecutionPolicy Bypass -File $masterRoute -Hint $c.hint -OutDir $tmp 2>&1
        $scope = Join-Path $tmp 'route-scope.md'
        if (Test-Path $scope) {
            $text = Get-Content $scope -Raw -Encoding UTF8
            if ($text -match 'primary:\s*(\S+)') { $got = $Matches[1] }
        }
    } catch { $got = 'EXC:' + $_.Exception.Message }
    finally { Remove-Item -Recurse -Force $tmp }
    if ($got -ne $c.expect) { 
        # Record failure - non-zero exit causes CI failure

        Write-Error "Case $($c.id): expected '$($c.expect)', got '$got'"
    }
}

Because this identical script runs on both Windows and Ubuntu via the matrix strategy, any platform-specific behavioral differences in master-route.ps1 or its dependencies immediately trigger a CI failure.

Routing Coherence and Supply-Chain Validation

Following the regression test, the pipeline validates structural integrity. In .github/workflows/ci.yml (lines 31‑34), the job executes verify-routing-coherence.ps1:

      - name: Routing coherence + supply‑chain pin gate
        shell: pwsh
        run: ./skills/scripts/verify-routing-coherence.ps1

This script checks the internal consistency of routing.json and validates supply-chain pin gates, ensuring that the routing manifest remains semantically valid across both operating systems. Like the regression test, it exits with a non-zero status on failure, causing the matrix job for that specific platform to report an error.

Summary

• Matrix Strategy: The CI uses a GitHub Actions matrix with windows-latest and ubuntu-latest to parallelize testing across operating systems.
• Shell Unification: A Linux shim symlinks pwsh to powershell, allowing identical PowerShell scripts to run on both platforms without modification.
• Regression Coverage: test-routing.ps1 validates 163 benchmark cases from routing-benchmark.json against the expected primary results.
• Integrity Checks: verify-routing-coherence.ps1 ensures the routing manifest and supply-chain pins remain consistent across platforms.
• Fail-Fast Behavior: Non-zero exits from either script cause immediate CI failure for that platform, preventing inconsistent routing logic from reaching production.

Frequently Asked Questions

What is the purpose of the PowerShell shim in the Ubuntu runner?

The shim ensures command compatibility between Windows and Linux CI runners. Because the test scripts invoke powershell (the traditional Windows executable name) but Ubuntu only provides pwsh (PowerShell Core), the workflow creates a symlink at /usr/local/bin/powershell pointing to the pwsh binary. This allows the same test code to execute unmodified on both platforms.

How many test cases does the routing regression suite include?

The regression suite includes 163 test cases defined in skills/tests/routing-benchmark.json. Each case specifies an input hint and an expected primary routing outcome, which skills/scripts/test-routing.ps1 validates by invoking master-route.ps1 and comparing the actual result against the expectation.

What happens when a routing test fails on one platform but passes on the other?

Because the CI uses a build matrix with independent jobs for each operating system, a failure on Ubuntu but not Windows (or vice versa) results in a failed status for the Ubuntu job while the Windows job passes. This immediately surfaces platform-specific regressions in the GitHub Actions interface, requiring developers to investigate OS-specific behavior in the routing logic before merging.

Which files are critical for the cross-platform CI routing validation?

The critical files include:

• .github/workflows/ci.yml: Defines the matrix strategy and execution steps.
• skills/scripts/test-routing.ps1: Executes the 163-case regression test.
• skills/scripts/verify-routing-coherence.ps1: Validates routing manifest integrity.
• skills/scripts/master-route.ps1: The central routing entry point under test.
• skills/tests/routing-benchmark.json: Contains the canonical expected results.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}