What Checks Does validate-skills.py Perform? A Complete Guide to Claude-Skills Validation

The validate-skills.py script performs comprehensive validation across YAML front-matter, markdown structure, reference files, workflow definitions, and cross-skill references to enforce repository quality standards.

The validate-skills.py utility serves as the central quality gate for the Jeffallan/claude-skills repository. This Python script orchestrates multiple specialized checker classes to validate every aspect of a skill's definition, from metadata syntax to documentation accuracy. Understanding what checks validate-skills.py performs helps contributors ensure their skills meet the project's strict specifications before submission.

YAML Front-Matter Validation

The script enforces strict schema compliance for every SKILL.md file's front-matter block through several specialized checkers defined in scripts/validate-skills.py (lines 69–132).

Syntax and Required Fields

• YamlChecker – Parses the front-matter block to guarantee syntactically valid YAML.
• RequiredFieldsChecker – Ensures name and description keys are present.
• NameFormatChecker – Verifies the skill name matches the regex ^[a-zA-Z0-9-]+$ and confirms the directory name mirrors the declared name.

Description Standards

• DescriptionLengthChecker – Warns if the description exceeds 1024 characters.
• DescriptionFormatChecker – Enforces the trigger-only prefix "Use when" to maintain consistent tone.

Metadata Schema

• MetadataFieldsChecker – Validates that a metadata map exists and contains all required sub-fields: triggers, role, scope, output-format, domain, and related-skills.
• MetadataEnumChecker – Includes ScopeEnumChecker and OutputFormatEnumChecker to warn on unknown scope or output-format values.

Markdown Body Structure Checks

Beyond metadata, validate-skills.py validates the document structure of skills/*/SKILL.md files (lines 85–128).

• CoreWorkflowStepCountChecker – Confirms the "## Core Workflow" section contains exactly five numbered steps.

• WhenToUseFormatChecker – Validates that the "## When to Use" section follows bullet-list formatting.

• SectionOrderChecker – Ensures H2 headings appear in the canonical order defined in the CANONICAL_SECTIONS constant.

• LineCountChecker – Enforces that the body (excluding front-matter) contains between 80 and 100 non-blank lines.

Reference File Validation

The script verifies the integrity of supplementary documentation in skills/*/references/*.md (lines 138–166).

• ReferencesDirectoryChecker – Confirms each skill contains a references/ folder.
• ReferenceFileCountChecker – Ensures at least one .md file exists within that folder.
• NonStandardHeadersChecker – Flags obsolete "Reference for:" or "Load when:" headers that violate the current Agent-Skills specification.

Workflow Definition Checks

For the commands/ directory, validate-skills.py performs deep validation of YAML workflow definitions and their manifests (lines 191–273).

• WorkflowDefinitionChecker – Validates each per-command YAML file for required fields, correct phase values for phased commands, enum values for status, requires, inputs, outputs, and verifies that referenced files actually exist.
• ManifestDagChecker – Parses commands/workflow-manifest.yaml to check phase definitions, description files, dependency graphs, duplicate commands, and runs a depth-first search (DFS) to detect directed acyclic graph (DAG) cycles.
• WorkflowOrphanChecker – Identifies command .md files that lack corresponding YAML definitions.

Cross-Skill Reference Validation

The script maintains graph integrity across the entire skill ecosystem (lines 445–530).

• CrossRefChecker – Builds a graph from metadata.related-skills and reports:
  • Bidirectional mismatches – Skill A references Skill B, but Skill B does not reference Skill A.
  • Orphan skills – Skills with no incoming or outgoing related-skill links.

Documentation Count Consistency

To prevent documentation drift, validate-skills.py includes a meta-validation check (lines 374–421).

• CountConsistencyChecker – Scans top-level documentation files (README.md, ROADMAP.md, QUICKSTART.md, etc.) and warns when declared counts of skills or reference files do not match the actual counts computed from the repository.

Running the Validator

Execute scripts/validate-skills.py from the repository root to invoke the validation suite.

Run All Checks

python scripts/validate-skills.py

Filter by Category

# Only YAML front-matter checks

python scripts/validate-skills.py --check yaml

# Only reference-file checks

python scripts/validate-skills.py --check references

# Only workflow definition and manifest checks

python scripts/validate-skills.py --check workflows

# Only cross-skill reference validation

python scripts/validate-skills.py --check crossrefs

Validate Single Skill

python scripts/validate-skills.py --skill react-expert

CI Pipeline Output

# JSON output for automated pipelines

python scripts/validate-skills.py --format json

The script exits with status 1 if any ERROR-level issue is detected, otherwise 0 (warnings do not trigger failure).

Summary

The validate-skills.py script enforces a comprehensive set of rules that keep the Claude-Skills repository clean, self-documenting, and CI-ready:

• YAML front-matter correctness – Validates syntax, required fields, naming conventions, description format, and metadata enums.
• Markdown body conventions – Enforces section order, specific step counts, line limits, and bullet formatting.
• Reference-file hygiene – Requires directories, minimum file counts, and flags obsolete headers.
• Workflow integrity – Validates command YAML schema, manifest DAG cycles, and orphan detection.
• Cross-skill linkage – Verifies bidirectional related-skills references and identifies orphans.
• Documentation count alignment – Ensures README and roadmap counts match actual repository contents.

Frequently Asked Questions

What is the validate-skills.py script used for?

The validate-skills.py script serves as the central quality gate for the Jeffallan/claude-skills repository. It validates YAML front-matter, markdown structure, reference files, workflow definitions, and cross-skill references to ensure every skill adheres to the project's strict specification before merging.

How do I run only specific checks in validate-skills.py?

Use the --check flag followed by the category name. Valid options include yaml (front-matter), references (reference files), workflows (command definitions), and crossrefs (cross-skill links). For example: python scripts/validate-skills.py --check yaml.

What happens if validate-skills.py finds errors?

The script exits with status code 1 if any ERROR-level issue is detected, making it suitable for CI/CD pipelines. Warnings do not trigger a failure exit code. When run interactively, it prints a human-readable table summarizing all issues; with --format json, it outputs machine-parseable results.

Where are the checker classes defined in the source code?

All checker classes are defined in scripts/validate-skills.py. YAML front-matter checkers appear around lines 69–132, markdown body checkers around lines 85–128, reference file checkers around lines 138–166, and workflow checkers around lines 191–273. The orchestration logic resides in the SkillValidator and main() functions toward the end of the file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}