How to Integrate SkillSpector into CI/CD Pipelines with Exit Codes and JSON Output

SkillSpector provides deterministic exit codes (0 for safe/caution, 1 for high risk, 2 for errors) and structured JSON output via --format json, enabling automated security gates in CI/CD workflows for AI agent skill packages.

The NVIDIA/SkillSpector repository delivers a security scanning CLI designed specifically for AI-agent skill packages. By leveraging machine-readable output and clear exit code contracts, teams can automate security decisions directly within their build pipelines without manual intervention.

Understanding SkillSpector Exit Codes

The skillspector scan command implements a deterministic exit code contract documented in the README under Integrating SkillSpector → Exit codes. This contract allows CI systems to distinguish between acceptable risks and critical security blocks.

Exit Code 0: Success

A return value of 0 indicates the scan completed successfully and the risk score is ≤ 50, corresponding to recommendations of SAFE or CAUTION. The pipeline may proceed with installation or deployment.

Exit Code 1: High Risk Detected

Exit code 1 signals the scan succeeded but detected a risk score > 50, triggering the DO_NOT_INSTALL recommendation. This code is designed to fail CI gates automatically when using set -e or equivalent pipeline configurations.

Exit Code 2: Scan Failure

Exit code 2 indicates a failure condition—invalid input paths, unreadable source files, or internal errors. This requires investigation into the scan configuration rather than the skill content itself.

Generating Machine-Readable JSON Output

SkillSpector generates structured JSON reports using the --format json flag, with the schema implemented in src/skillspector/models.py through the Finding.to_dict() serialization method. Output redirects to stdout by default or to a file via -o/--output.

The JSON payload includes:

{
  "skill": { "name": "example-skill", "source": "./path", "scanned_at": "2024-01-15T10:30:00Z" },
  "risk_assessment": { 
    "score": 25, 
    "severity": "LOW", 
    "recommendation": "SAFE" 
  },
  "components": [],
  "issues": [],
  "metadata": {
    "has_executable_scripts": false,
    "skillspector_version": "0.1.0",
    "llm_requested": true,
    "llm_available": true
  }
}

The risk_assessment.recommendation field provides the specific verdict: SAFE, CAUTION, or DO_NOT_INSTALL.

Step-by-Step CI/CD Integration

Install the CLI

Add SkillSpector to your pipeline environment using uv or pip. For CLI-only functionality:

uv tool install git+https://github.com/NVIDIA/SkillSpector.git

For MCP server extras:

uv tool install 'skillspector[mcp] @ git+https://github.com/NVIDIA/SkillSpector.git'

Run the Scan

Execute the scan with JSON formatting and artifact capture:

skillspector scan ./my-skill/ \
  --format json \
  --output skill_report.json

Gate Based on Exit Codes

Implement basic gating using shell exit code checks:

if ! skillspector scan ./my-skill/ --format json -o skill_report.json; then
    echo "⚠️  Skill rejected – see skill_report.json"
    exit 1
fi
echo "✅  Skill passed (score ≤ 50)"

Fine-Grained Parsing with JSON

For nuanced policies that distinguish between SAFE and CAUTION, parse the JSON using jq:

RECOMMENDATION=$(jq -r '.risk_assessment.recommendation' skill_report.json)

case "$RECOMMENDATION" in
  SAFE)   echo "✅  Safe – continue";;
  CAUTION) echo "⚠️  Caution – requires manual review";;
  DO_NOT_INSTALL) echo "❌  Blocked – failing build"; exit 1;;
esac

Complete GitHub Actions Example

The following workflow demonstrates full integration with artifact upload and conditional gating:

name: CI – Skill Safety Check
on: [push, pull_request]

jobs:
  skillspector:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install uv
        run: curl -LsSf https://github.com/astral-sh/uv/releases/latest/download/uv-installer.sh | sh

      - name: Install SkillSpector CLI
        run: uv tool install git+https://github.com/NVIDIA/SkillSpector.git

      - name: Run SkillSpector scan
        id: scan
        run: |
          skillspector scan ./my-skill/ \
            --format json \
            --output skill_report.json
        continue-on-error: true

      - name: Upload report artifact
        uses: actions/upload-artifact@v4
        with:
          name: skill-report
          path: skill_report.json

      - name: Enforce gate based on exit code
        if: steps.scan.outcome != 'success'
        run: |
          echo "❌ SkillSpector detected high-risk issues"
          exit 1

      - name: Optional CAUTION handling
        run: |
          RECOMMEND=$(jq -r '.risk_assessment.recommendation' skill_report.json)
          if [ "$RECOMMEND" = "CAUTION" ]; then
            echo "⚠️  CAUTION – flagging for manual review"
          fi

Key Source Files for Reference

• src/skillspector/cli.py: Contains the CLI entry point, argument parsing for --format and --output, and exit code logic.
• src/skillspector/models.py: Defines the Finding data model and the to_dict() method used for JSON serialization.
• README.md: Documents the exit code table and JSON schema specifications.
• docs/SUPPRESSION.md: Provides baseline configuration for managing false positives in CI environments.
• pyproject.toml: Declares package metadata and the skillspector[mcp] optional dependency.

Summary

• Exit codes 0, 1, and 2 provide deterministic signals for SAFE/CAUTION, DO_NOT_INSTALL, and error states respectively.
• --format json generates machine-readable reports via Finding.to_dict() in src/skillspector/models.py.
• CI integration follows a "run → check → gate" pattern using standard shell exit code handling or JSON parsing with tools like jq.
• Artifact persistence allows downstream pipeline steps to access detailed findings even after the scan step completes.

Frequently Asked Questions

What exit code does SkillSpector return for high-risk skills?

SkillSpector returns exit code 1 when the risk score exceeds 50, indicating a DO_NOT_INSTALL recommendation. This differs from exit code 2, which indicates scan failures such as invalid input or internal errors.

How do I parse SkillSpector JSON output in a CI pipeline?

Use command-line JSON processors like jq to extract specific fields. For example, jq -r '.risk_assessment.recommendation' skill_report.json retrieves the recommendation string (SAFE, CAUTION, or DO_NOT_INSTALL) for conditional logic.

Can SkillSpector fail a build automatically?

Yes. By running skillspector scan without continue-on-error settings (or by checking exit codes explicitly), the pipeline will fail automatically when the tool returns exit code 1 (high risk) or 2 (scan error), compatible with set -e shell configurations.

Where is the JSON output schema defined?

The JSON structure is generated by the Finding.to_dict() method in src/skillspector/models.py, with high-level schema documentation located in the README.md under the Machine-readable output section. The schema includes skill metadata, risk assessment scores, and detected issues.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}