How AI-Powered Policy Scanning Works for Claude Plugins: A Complete Technical Breakdown

The AI-powered policy scanning in anthropics/claude-plugins-community works through a two-stage GitHub Action called scan-plugins that combines deterministic static analysis with Claude model evaluation to enforce safety and security policies.

Every plugin submitted to the Claude community marketplace must pass automated policy validation before merging. This system, implemented in the anthropics/claude-plugins-community repository, fuses static pin checking with large language model reasoning to catch both supply-chain risks and nuanced security violations that traditional analysis would miss.

The Two-Stage Scanning Architecture

The scan-plugins action orchestrates two coordinated stages:

• Static Pin Check – Deterministic Bash/JQ analysis of package specifications to detect floating (unpinned) runtime launchers
• Claude Policy Scan – Model invocation that reads plugin source files and renders a policy verdict

This dual-layer design ensures deterministic safety guarantees while leveraging AI for interpretive policy enforcement.

Stage 1: Static Pin Check (Deterministic Analysis)

The first stage runs entirely in scripts/static-pin-check.sh. It analyzes how plugins specify their execution environment—particularly npx -p, uvx, bunx, and similar package launchers.

What Gets Checked

Launcher Pattern	Risk if Unpinned
npx -p package@version ✅	Pinned to known version
npx -p package ❌	Floating—fetches latest at runtime
uvx tool@version ✅	Pinned, reproducible
uvx tool ❌	Floating, supply-chain drift risk

The pin_check_entry_waived function in lib/pin-check.sh parses CLI flags to extract package specifications. When a floating spec is detected, the entry is flagged with unpinned_autoexec_runtime=true for inclusion in the final report.

This stage is fully deterministic—no AI invocation, no network calls to external models, pure shell and jq processing.

Stage 2: Claude Policy Scan (Model Evaluation)

The second stage in scripts/scan.sh invokes the Claude CLI with a curated policy prompt that directs the model to examine plugin source code.

Model Invocation Parameters

claude -p "$prompt" \
       --max-tokens 4096 \
       --temperature 0 \
       --model claude-3-5-sonnet-20240620 \
       --file-tools-readonly \
       --no-stream

The -p flag provides the combined prompt (policy + read instructions). The --file-tools-readonly constraint ensures the model cannot modify files—critical for CI security.

Files the Model Reads

Per policy/prompt.md, Claude examines:

• .claude-plugin/plugin.json – Plugin metadata and capabilities
• .mcp.json – Model Context Protocol configuration
• skills/ – Skill definitions and implementations
• agents/ – Agent configurations
• commands/ – CLI command handlers
• hooks/ – Lifecycle hook implementations

Verdict Structure

The model returns a JSON object with these fields:

{
  "passes": false,
  "summary": "Plugin uses disallowed network endpoint",
  "violations": ["External API calls to unapproved domain"],
  "may_make_external_network_calls": true,
  "may_download_additional_software": false
}

The may_* risk flags enable downstream automation to apply appropriate sandboxing or approval workflows.

Target Resolution and Workflow Integration

Finding What to Scan

The resolve_scan_targets function in lib/targets.sh determines which marketplace entries require scanning. It supports two modes:

• PR-scoped (default): Only plugins modified in the current pull request
• Full scan (scan-all-external=true): Every external marketplace entry

Targets are written to a JSON file consumed by both scanning stages.

Result Aggregation and CI Integration

Lines 139–161 of scan.sh merge static-pin results with the model's JSON verdict. The combined output includes:

• scanned – Complete array of verdict objects
• failed – Names of plugins that did not pass
• pin-scanned – Raw static-pin results for bump-plugin-shas workflow

When fail-on-findings=true, violations trigger GitHub error annotations:

::error file=plugin.json,line=1::scan-plugins: my-plugin FAILS policy — uses disallowed network endpoint

Using the scan-plugins Action

Basic Workflow Configuration

# .github/workflows/scan-plugins.yml

name: Scan Plugins
on:
  pull_request:
    paths:
      - '.claude-plugin/marketplace.json'

jobs:
  policy-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Scan changed plugins
        uses: anthropics/claude-plugins-community/.github/actions/scan-plugins@v1
        with:
          marketplace-path: .claude-plugin/marketplace.json
          fail-on-findings: true
          scan-all-external: false

Available Inputs

Input	Type	Default	Description
marketplace-path	string	(required)	Path to marketplace.json
fail-on-findings	boolean	false	Fail CI when violations found
scan-all-external	boolean	false	Scan all plugins vs. PR-changed only

Output Format

The action produces a step summary table:

| plugin-name | ✅ passes | ⚠️ may_make_external_network_calls | may_download_additional_software | unpinned_autoexec_runtime | Summary... |

Key Implementation Files

Understanding the complete AI-powered policy scanning system requires familiarity with these source files in .github/actions/scan-plugins/:

• scripts/scan.sh – Core orchestration script (lines 111–124 for Claude invocation, 139–161 for result merging)
• scripts/static-pin-check.sh – Deterministic package spec analysis
• policy/prompt.md – The policy instructions provided to Claude
• lib/targets.sh – Target resolution logic (resolve_scan_targets)
• lib/pin-check.sh – Helper for parsing pinned vs. floating specs (pin_check_entry_waived)
• action.yml – GitHub Action interface definition
• ../../workflows/validate-plugins.yml – Production CI workflow integrating the scan

Design Rationale: Why This Architecture?

The fusion of static and AI-powered analysis addresses distinct failure modes:

Deterministic static checks guarantee that runtime-installable binaries are pinned to cryptographic identities. This prevents supply-chain attacks where a compromised registry publishes malicious updates.

Model-level reasoning catches policy violations requiring semantic interpretation: disallowed system calls, insecure network endpoint patterns, overly broad file system access, or subtle data exfiltration vectors. Static analysis of plugin.json cannot determine whether an HTTP client will be used to exfiltrate user data—Claude can.

Temperature-zero sampling (--temperature 0) ensures reproducible verdicts for identical inputs, making the AI component deterministic enough for CI gates.

Summary

• Two-stage validation: Static pin checking (static-pin-check.sh) plus Claude policy evaluation (scan.sh) provides defense in depth
• Deterministic safety: Unpinned package launchers are caught before any model invocation
• Semantic policy enforcement: Claude interprets source code against policy/prompt.md criteria that static tools cannot evaluate
• CI-native integration: The scan-plugins action emits GitHub annotations and fails builds via fail-on-findings
• Read-only constraints: --file-tools-readonly and --no-stream minimize attack surface in automated execution

Frequently Asked Questions

What triggers the AI-powered policy scan?

The scan runs automatically on pull requests that modify .claude-plugin/marketplace.json via the validate-plugins.yml workflow. Set scan-all-external: true to scan every marketplace entry regardless of PR changes.

Why does the scan use both static analysis and AI evaluation?

Static analysis in static-pin-check.sh provides guaranteed, fast detection of supply-chain risks (unpinned packages). The Claude model evaluation handles security patterns requiring interpretation—network behavior analysis, permission scope review, and policy compliance that regex or AST parsing cannot reliably determine.

Can the policy scan be run locally?

Yes. The underlying scripts in .github/actions/scan-plugins/scripts/ can execute independently with claude CLI installed and ANTHROPIC_API_KEY configured. The resolve_scan_targets function accepts manual target specification for local debugging.

What Claude model version performs the policy scan?

The action pins claude-3-5-sonnet-20240620 with --temperature 0 for reproducible, cost-effective evaluation. This is hardcoded in scan.sh lines 118–120 and can be overridden by modifying the script for testing newer models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}