How to Validate Skills Before Submitting a Pull Request: The 8-Step CI Pipeline

When you submit a pull request modifying any SKILL.md file in sickn33/antigravity-awesome-skills, an automated eight-step validation pipeline checks your skill against nine specific quality criteria including front-matter schema, required sections, and security disclaimers before the PR can be merged.

The sickn33/antigravity-awesome-skills repository enforces strict quality standards through automated CI pipelines. Understanding the skill validation process before submitting a pull request helps contributors pass automated checks on the first attempt and avoid revision cycles.

The 8-Step Validation Pipeline

1. Workflow Trigger on SKILL.md Changes

When a PR modifies any SKILL.md file, GitHub Actions triggers two workflows defined in .github/workflows/skill-review.yml and .github/workflows/ci.yml. The pull_request event uses the path filter **/SKILL.md to target only skill-related changes.

2. Repository Checkout and Environment Setup

The CI job checks out the PR code using actions/checkout@v4, then installs Node.js 14+ and Python 3.10. The environment adds pyyaml as a dependency to parse skill front-matter.

3. Directory Structure Verification

A shell test validates that required top-level directories exist, including skills/ and apps/web-app/. This ensures the repository structure remains intact before proceeding to content validation.

4. The Nine-Point Skill Validation Script

The core validation runs via npm run validate, which executes tools/scripts/validate_skills.py through a thin Node wrapper located at tools/scripts/run-python.js. This script performs nine specific checks on every modified SKILL.md file:

• Front-matter presence and YAML syntax: Validates that the file starts with valid YAML delimiters (---) and that the content parses correctly using pyyaml.

• Required metadata fields: Ensures name, description, risk, and source fields exist in the front-matter.

• Folder name consistency: Verifies that the name field matches the containing folder name.

• Description constraints: Checks description length and data type.

• Risk level validation: Confirms the risk value belongs to the allowed set defined by the project.

• Date format verification: Validates optional date_added fields match the <YYYY-MM-DD> format.

• "When to Use" section: Detects the presence of a ## When to Use header in the markdown body.

• Offensive skill disclaimer: For skills marked as offensive, checks for the required security disclaimer containing "AUTHORIZED USE ONLY".

• Link integrity: Scans for dangling local markdown links that reference non-existent files.

5. Strict Mode and Quality Bar Enforcement

While the default CI run operates in non-strict mode, the pipeline enforces the Quality Bar Checklist using npm run validate:strict. This command adds the --strict flag to the Python script, converting validation warnings into fatal errors. Any missing optional section or formatting warning will fail the build in this mode.

6. References Validation (Conditional)

If the PR metadata indicates requires_references (determined by tools/lib/workflow-contract.js), the workflow runs npm run validate:references. This step ensures all cited external sources in the skill documentation exist and are accessible.

7. Test Suite and Security Scans

After skill-specific validation, the pipeline executes npm run test to run unit tests and npm run security:docs to scan documentation for unsafe content or security regressions.

8. Review Feedback and PR Comments

If any validation step fails, the job outputs detailed logs specifying exactly which skill file broke which rule. The skill-review action posts a summary comment directly to the PR, giving immediate feedback. Contributors must fix the reported issues and push new commits. Only when all validation steps pass does the PR satisfy the "Quality Bar" and become eligible for merge.

Local Validation Before Submitting

Contributors can run the exact same validation checks locally to catch errors before opening a PR.

Standard Mode Validation

Install dependencies and run the validator:

npm ci
pip install pyyaml

npm run validate

Strict Mode Validation (CI-Equivalent)

To match the CI behavior exactly:

npm run validate:strict

This treats warnings such as missing ## When to Use sections as fatal errors.

Validation Script Implementation Details

The parse_frontmatter function in tools/scripts/validate_skills.py handles YAML extraction:

def parse_frontmatter(content, rel_path=None):
    """Parse frontmatter using PyYAML for robustness."""
    fm_match = re.search(r'^---\s*\n(.*?)\n---', content, re.DOTALL)
    if not fm_match:
        return None, ["Missing or malformed YAML frontmatter"]
    fm_text = fm_match.group(1)
    try:
        metadata = yaml.safe_load(fm_text) or {}
        # … additional validation …

        return dict(metadata), fm_errors
    except yaml.YAMLError as e:
        return None, [f"YAML Syntax Error: {e}"]

Sample CI Output

When validation fails, contributors see specific error messages:

⚠️  my-skill/SKILL.md: Missing '## When to Use' section

❌  my-skill/SKILL.md: Description is oversized (345 chars). Must be concise.
❌  my-skill/SKILL.md: OFFENSIVE SKILL MISSING SECURITY DISCLAIMER! (Must contain 'AUTHORIZED USE ONLY')

Key Files in the Validation Architecture

Understanding the repository's validation infrastructure helps with debugging:

• tools/scripts/validate_skills.py: Core Python validator enforcing front-matter schema, risk levels, content sections, and link integrity.
• .github/workflows/ci.yml: Orchestrates the full CI pipeline including environment setup, test execution, and quality checklist enforcement.
• .github/workflows/skill-review.yml: Triggers the tesslio/skill-review GitHub Action on PRs touching SKILL.md files.
• package.json: Defines npm shortcuts including validate, validate:strict, test, and security:docs.
• tools/scripts/run-python.js: Node.js wrapper that executes Python validators from npm scripts.
• tools/lib/workflow-contract.js: Defines PR metadata contracts such as requires_references consumed by CI logic.

Summary

• The validation pipeline triggers automatically when PRs modify SKILL.md files, running eight distinct steps from checkout to security scans.
• Nine specific checks validate front-matter YAML, required fields, folder naming, content sections, and security disclaimers through tools/scripts/validate_skills.py.
• Strict mode (npm run validate:strict) enforces the Quality Bar by treating warnings as fatal errors, matching CI behavior.
• Contributors should run npm run validate locally before submitting to catch formatting errors, oversized descriptions, or missing sections early.
• The pipeline only passes when all checks succeed, at which point the skill-review action confirms eligibility for merge.

Frequently Asked Questions

What happens if my skill fails validation in CI?

The workflow logs display specific error messages indicating which file failed and which rule was violated—such as missing front-matter fields or an oversized description. The skill-review action also posts a summary comment to your PR. You must fix the reported issues, commit the changes, and push them to the PR branch to retrigger validation.

Can I run the validation checks on my local machine?

Yes. Install Node.js 14+, Python 3.10, and pyyaml, then run npm ci followed by npm run validate for standard checks or npm run validate:strict to match the CI's strict mode. This runs the same tools/scripts/validate_skills.py script used in the pipeline via the Node wrapper at tools/scripts/run-python.js.

What is the difference between standard mode and strict mode validation?

Standard mode reports warnings (like a missing ## When to Use section) as non-fatal alerts, while strict mode—activated with the --strict flag via npm run validate:strict—converts every warning into a fatal error. The CI pipeline uses strict mode to enforce the Quality Bar Checklist, ensuring all skills meet the repository's highest standards before merging.

How does the CI know if my skill requires reference validation?

The system checks PR metadata defined in tools/lib/workflow-contract.js for the requires_references flag. If this condition is true, the workflow conditionally executes npm run validate:references to verify all cited sources exist. Most standard skills do not trigger this step unless explicitly configured.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}