Static Tests Performed by the validate-plugins.yml Workflow in Claude Plugins Community

The validate-plugins.yml workflow executes six network-free static test suites—invariant validation (I1-I11), freeze/SHA-exempt handling, manifest synthesis, owner liveness sweeping, external manifest resolution, and pin-check golden vectors—to verify marketplace.json integrity without external network calls.

The anthropics/claude-plugins-community repository maintains marketplace quality through rigorous automated validation. The validate-plugins.yml workflow orchestrates pure Bash static tests that operate on synthetic marketplace.json fixtures, ensuring all plugin metadata meets strict structural and safety standards before merging.

Overview of the Static Validation Pipeline

The validation workflow located at .github/workflows/validate-plugins.yml runs a suite of pure-static tests that require no network access. Each test suite is implemented as a separate Bash script under .github/actions/, using jq for JSON processing and synthetic fixtures to simulate marketplace states.

All tests execute in isolation against controlled inputs, allowing the pipeline to detect regressions in validation logic itself while guaranteeing that the production marketplace.json adheres to invariant requirements.

Invariant Validation (I1-I11)

The foundational static test suite validates eleven structural invariants defined for the marketplace schema.

Test Implementation

The script .github/actions/validate-plugins/test-invariants.sh verifies that marketplace.json entries satisfy sorting requirements, unique naming constraints, description length limits, safe URL protocols, SHA presence, path safety, hidden Unicode detection, and name formatting rules.

This suite also tests diff-scoping behavior, ensuring that validation errors demote to warnings for unchanged entries during incremental updates.

Code Example: Unsafe URL Detection

The following excerpt demonstrates how invariant I4 validates URL security:

# Verify that a malformed URL triggers I4

f=$(mk i4 <<'EOF'
{"plugins":[{"name":"abc","description":"ten chars ok","source":{"source":"url","url":"http://insecure.example/x","sha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}}]}
EOF
); assert_fires "I4 unsafe url" I4 "$f"

This fixture creates a plugin entry with an insecure http:// URL and asserts that the validator correctly fires invariant I4.

Freeze and SHA-Exempt Handling

The freeze mechanism ensures that specific plugin versions remain pinned regardless of upstream changes.

Test Coverage

The .github/actions/bump-plugin-shas/test-bump.sh script validates the freeze-shas and sha-exempt mechanisms. It confirms that frozen pins are held static, whole-word matching functions correctly, malformed entries trigger appropriate warnings, and that guard ordering prioritizes freeze rules over exempt rules.

Freeze Validation Example

# A frozen, pinned entry is held and recorded

f=$(mk freeze <<'EOF'
{"plugins":[{"name":"frozen-plugin","source":{"url":"https://github.com/acme/frozen-plugin","sha":"1111111111111111111111111111111111111111"}}]}
EOF
)
FREEZE_SHAS_FIXTURE="frozen-plugin"; SHA_EXEMPT_FIXTURE=""
run_bump "$f"
assert_reason "frozen-plugin" "frozen at current pin (freeze-shas)" "freeze fires + recorded in skipped[]"

This test verifies that plugins listed in freeze-shas are correctly skipped during bump operations and recorded in the skipped[] array.

Manifest Synthesis Verification

The workflow tests the temporary manifest generation used during SHA bumping operations.

Synthesis Correctness

The .github/actions/bump-plugin-shas/test-bump-manifest.sh script confirms that the manifest-synthesis step produces valid JSON structures. It validates that regenerated manifests match the original structure when no changes are required, ensuring idempotency in the synthesis pipeline.

Owner Liveness Sweep

Stale ownership data can compromise marketplace security.

Sweep Validation

The .github/actions/owner-liveness-sweep/test-sweep.sh implements tests for the owner-liveness sweep. This static test exercises the logic that scans marketplace entries to identify and report owners that have disappeared or changed credentials, ensuring the sweep correctly flags stale ownership records without requiring live network queries.

External Manifest Resolution

Plugins referencing external manifests require specialized validation logic.

Resolution Testing

The .github/actions/validate-plugins/test-external-manifest.sh validates external manifest handling. It ensures that plugins referencing external manifest URLs are correctly resolved and that structural errors in external manifests are caught during the static validation phase.

Pin-Check Golden Vectors

The final static test suite guarantees SHA pin accuracy through comparative analysis.

Golden Vector Testing

The .github/actions/scan-plugins/test-pin-check.sh runs pin-check golden vectors, comparing current plugin SHAs against known-good "golden" expectations. This regression test guarantees that pin reporting remains accurate and detects any drift in hash calculation logic.

Workflow Orchestration

The .github/workflows/validate-plugins.yml file coordinates execution of all six test suites in sequence. After the Bash-based static tests complete, the workflow runs the official validate-plugins action against the actual marketplace to ensure real-world validation logic aligns with the static expectations.

All test scripts operate network-free, relying solely on Bash, jq, and synthetic fixtures to validate behavior.

Summary

• Invariant Validation (I1-I11): Enforces structural rules, safe URLs, and diff-scoping via test-invariants.sh
• Freeze/SHA-Exempt Logic: Validates pinning mechanics via test-bump.sh
• Manifest Synthesis: Confirms JSON regeneration accuracy via test-bump-manifest.sh
• Owner Liveness: Tests stale ownership detection via test-sweep.sh
• External Manifests: Checks external reference resolution via test-external-manifest.sh
• Pin-Check Vectors: Runs SHA regression tests via test-pin-check.sh

Frequently Asked Questions

What makes these tests "static"?

These tests are classified as static because they execute without network access, operating entirely on synthetic marketplace.json fixtures stored in the repository. According to the anthropics/claude-plugins-community source code, the test suite uses Bash and jq to validate logic paths without querying external URLs, GitHub APIs, or remote manifests.

Why are the invariants numbered I1 through I11?

The I1-I11 numbering scheme represents the eleven specific validation rules defined for marketplace.json structure. Each invariant targets a distinct quality gate: I1 checks alphabetical sorting, I4 validates HTTPS URLs, and others enforce unique naming, description lengths, and Unicode safety. The test-invariants.sh script maps each invariant to a specific test fixture.

How does the workflow handle unchanged entries during validation?

The static tests implement diff-scoping logic that demotes hard errors to warnings when validation failures occur in marketplace entries that were not modified in the current pull request. The test-invariants.sh script specifically tests this behavior to ensure incremental updates don't fail due to pre-existing issues in unchanged plugins.

What is the difference between freeze-shas and sha-exempt?

Freeze-shas explicitly pins specific plugins to their current SHA regardless of upstream changes, while sha-exempt removes SHA requirements entirely for certain entries. The test-bump.sh script validates that freeze rules take precedence over exempt rules in the guard ordering, ensuring frozen plugins remain pinned even when exempt patterns might otherwise apply.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}