How the `validate-plugins` Workflow Tests Plugin Entries in the Claude Plugins Community Repository

The validate-plugins workflow runs six distinct test suites—covering static invariants, SHA bumping, owner liveness, external manifests, pin-checking, and final live validation—to ensure every Claude plugin entry meets strict quality and security standards before merge.

The anthropics/claude-plugins-community repository uses a comprehensive continuous‑integration pipeline to guard the integrity of its plugin marketplace. The validate-plugins pipeline combines synthetic fixture tests with live manifest validation, reusing the same core logic exposed by the public Validate‑Plugins GitHub Action.

Overview: Six Layers of Validation

The workflow defined in .github/workflows/validate-plugins.yml executes a layered testing strategy. Each layer targets a specific failure mode, from malformed JSON to stale plugin owners.

Step	Test Suite	Purpose
1	Static invariant tests	Verify 11 hard‑coded rules (I1–I11) on synthetic data
2	Bump‑plugin‑SHAs tests	Validate SHA freezing and manifest regeneration
3	Owner‑liveness‑sweep tests	Confirm stale‑owner detection works correctly
4	External manifest resolution tests	Ensure the action resolves external marketplace files
5	Pin‑check golden vectors	Verify scan‑plugins pin‑checking logic
6	Final live validation	Run the action against the real marketplace.json

Static Invariant Tests (Step 1)

The first and most extensive suite lives in .github/actions/validate-plugins/test-invariants.sh. This script builds synthetic marketplace.json fixtures and pipes them to scripts/11-validate-invariants.sh.

The invariant validator enforces eleven strict rules:

• I1: Entries are alphabetically ordered by name
• I2: No duplicate plugin names exist
• I3: Description length falls within [10, 2000] characters
• I4: URLs use safe, allowed schemes and domains
• I5: Source SHAs are present and 40‑character hexadecimal
• I6: Per‑file naming conventions are followed
• I7: Vendored source files exist where declared
• I8: Names contain no dangerous shell characters
• I9: Hidden Unicode characters are absent
• I10: Name format matches the required pattern
• I11: Additional structural constraints

Each invariant violation triggers a GitHub‑compatible annotation. In scripts/11-validate-invariants.sh, the flag function emits errors for new violations:

printf '::error %s::invariant %s: %s\n' \
  "file=$MARKETPLACE_PATH,line=12" "I3" \
  "abc: description length 5 not in [10,2000]"

Diff‑Scoping for Pragmatic Enforcement

The script supports SCOPE_ERRORS_TO_CHANGED mode. When enabled, violations in unchanged entries downgrade to warnings to avoid blocking unrelated pull requests:

printf '::warning %s::invariant %s: %s\n' \
  "file=$MARKETPLACE_PATH,line=12" "I5" \
  "def: source.sha is missing or not a 40-char hex SHA [unchanged entry — not introduced by this PR; downgraded to warning]"

This mechanism prevents legacy defects from freezing the repository while maintaining strict gates for new contributions.

The test driver in test-invariants.sh invokes the validator:

# Inside .github/actions/validate-plugins/test-invariants.sh

run_invariants "$fixture_path" "$entries_dir"
bash scripts/11-validate-invariants.sh 2>&1 || true

SHA Bump and Manifest Tests (Step 2)

The workflow validates the bump‑plugin‑shas mechanism through two complementary scripts. .github/actions/bump-plugin-shas/test-bump.sh verifies that the "freeze‑shas" logic correctly skips pinned SHAs or updates them when explicitly requested. .github/actions/bump-plugin-shas/test-bump-manifest.sh confirms that post‑bump manifests still satisfy all invariants.

This ensures version bumps cannot accidentally relax validation requirements.

Owner Liveness Sweep Tests (Step 3)

Stale plugin ownership creates maintenance risk. The workflow runs .github/actions/owner-liveness-sweep/test-sweep.sh to exercise a synthetic sweep of plugin owners. The test confirms that:

• Truly stale owners are correctly flagged
• Active owners remain unaffected
• The sweep logic does not introduce false positives on unrelated PRs

External Manifest Resolution Tests (Step 4)

Not all consumers use the default marketplace.json location. .github/actions/validate-plugins/test-external-manifest.sh constructs temporary external marketplace files and validates that the action correctly handles:

• Custom entries_dir paths
• The skip-local-folders flag for remote‑only validation

Pin‑Check Golden Vectors (Step 5)

SHA pinning integrity is tested via .github/actions/scan-plugins/test-pin-check.sh. This golden‑vector test exercises the scan‑plugins action's pin‑checking logic against known‑good and known‑bad inputs, ensuring missing or malformed SHAs are reliably detected.

Final Live Validation (Step 6)

After all synthetic tests pass, the workflow invokes the Validate‑Plugins action itself against the repository's actual data:

- uses: ./.github/actions/validate-plugins
  with:
    marketplace-path: .claude-plugin/marketplace.json
    skip-local-folders: "true"
    scope-errors-to-changed: "true"

This final step ensures the same validation code tested in isolation now runs in production‑equivalent conditions. The scope-errors-to-changed input preserves the pragmatic diff‑scoping behavior, while skip-local-folders optimizes validation for the community repository's structure.

Core Validation Architecture

The Validate‑Plugins action is defined in .github/actions/validate-plugins/action.yml. It exposes reusable inputs that other repositories can consume, wrapping the invariant scripts and providing consistent behavior across the ecosystem.

The validation pipeline's design emphasizes test‑reusability: every major script has a corresponding test-*.sh driver that exercises edge cases without requiring live repository data. This enables confident refactoring of the core invariants while maintaining behavioral guarantees.

Summary

• The validate-plugins workflow runs six specialized test suites before any merge can complete.
• Static invariants (I1–I11) enforce structural, security, and quality rules via scripts/11-validate-invariants.sh.
• Diff‑scoping (SCOPE_ERRORS_TO_CHANGED) allows pragmatic enforcement that ignores pre‑existing defects.
• Synthetic fixtures enable comprehensive testing of SHA bumps, owner sweeps, and external manifests without live data risks.
• The final live validation step runs the public Validate‑Plugins action against the real marketplace.json, ensuring production‑ready behavior.

Frequently Asked Questions

What triggers the validate-plugins workflow?

The workflow runs on every pull request that modifies plugin‑related files. It is defined in .github/workflows/validate-plugins.yml and executes automatically via GitHub Actions.

Can I run the invariant tests locally?

Yes. The test scripts in .github/actions/validate-plugins/ are self‑contained Bash files. You can execute test-invariants.sh directly to validate synthetic fixtures without invoking the full CI pipeline.

How does diff‑scoping prevent unrelated PRs from being blocked?

When scope-errors-to-changed is enabled, the validator compares each failing entry against the PR's changed files. Violations in untouched entries emit ::warning annotations instead of ::error, allowing the workflow to succeed while still surfacing issues for maintainers.

What happens if a new invariant needs to be added?

Add the invariant logic to scripts/11-validate-invariants.sh following the existing I1–I11 pattern. Create corresponding test cases in test-invariants.sh to exercise both success and failure paths. Finally, update the invariant documentation in any README or schema files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}