Validation Pipeline for Claude Code Skills: A Complete Technical Guide

The Jeffallan/claude-skills repository implements a rigorous three-stage validation pipeline that checks YAML front-matter, workflow definitions, and Markdown syntax to ensure every skill conforms to the Agent Skills specification before release.

The validation pipeline for Claude Code skills is the quality gatekeeper of the Jeffallan/claude-skills repository. Before any skill reaches production, it must pass through three distinct validation stages that verify structural integrity, metadata compliance, and documentation correctness. This automated system prevents malformed skills from breaking the Claude Code workflow engine while enforcing consistent documentation standards across the entire repository.

Overview of the Three-Stage Validation Pipeline

The validation architecture separates concerns into three distinct phases. Each stage targets a specific layer of the skill ecosystem: the skill definition itself, the workflow integration layer, and the documentation format. The pipeline is implemented primarily in scripts/validate-skills.py and scripts/validate-markdown.py, with the former handling stages one and two, and the latter handling stage three.

Stage 1: Skill-File Validation

The first stage performs deep inspection of every SKILL.md file within the skills/ directory. This is the most comprehensive validation layer, ensuring that each skill definition is syntactically valid and semantically complete.

YAML Front-Matter and Metadata Checks

Every skill file must begin with valid YAML front-matter delimited by triple dashes. The YamlChecker class in scripts/validate-skills.py extracts this front-matter using BaseChecker._extract_frontmatter and validates the following required top-level fields:

• name: Must match the regex pattern ^[a-zA-Z0-9-]+$ and contain only alphanumeric characters and hyphens
• description: Must begin with the prefix "Use when" and not exceed 1024 characters in length
• metadata: A required block containing specific sub-fields including triggers, role, scope, output-format, domain, and related-skills

The ScopeEnumChecker and related classes validate that scope and output-format values belong to predefined enumerations defined in the Agent Skills specification.

Content Structure and Formatting Rules

Beyond metadata, the validation enforces strict document structure. The SectionOrderChecker verifies that SKILL.md files follow the canonical section order defined in the specification. Content formatting rules include:

• Line count constraints: Total non-blank lines must fall between 80 and 100 lines
• Reference material requirements: The references/ directory must contain at least one file and all referenced paths must resolve to actual files

Core Workflow Validation

The CoreWorkflowStepCountChecker specifically validates the "Core Workflow" section, ensuring it contains exactly five numbered steps. The "When to Use" section must be formatted as a bullet list. Any deviation from these structural requirements generates a ValidationIssue with severity level ERROR or WARNING.

Stage 2: Workflow Definition Validation

The second stage shifts focus from individual skills to the workflow integration layer, validating the command definitions and manifest files that orchestrate skill execution.

Command Definition Requirements

Each YAML file in the commands/ directory undergoes inspection by the WorkflowDefinitionChecker. This validator ensures every command definition contains:

• Required keys: command, path, description, inputs, outputs, and requires
• Phase-specific fields: Non-utility commands must include a phase field indicating their execution stage
• Enum compliance: Values for phase, status, requires, and input/output types must match known enumerations
• Path resolution: The path and description fields must resolve to real files within the repository

DAG Cycle Detection in Workflow Manifest

The commands/workflow-manifest.yaml serves as the central orchestration definition. The ManifestDagChecker validates this file using _detect_cycles, which implements a depth-first search (DFS) algorithm to detect circular dependencies in the phase graph. If a cycle is detected (for example, intake -> planning -> intake), the validator reports a DAG cycle detected error and the pipeline fails.

Stage 3: Markdown Sanity Checks

The final stage ensures documentation readability and prevents rendering errors that could break the Claude Code interface or GitHub's Markdown parser.

Code Block and Table Validation

The scripts/validate-markdown.py script performs lightweight structural analysis on all Markdown files in the skills/ directory. The validate_file() function walks each line to track code-block state and detect:

• Unclosed fenced code blocks: Tracks opening triple backticks (```) to ensure every code block is properly closed
• Malformed tables: Validates that table headers are followed by proper separator rows (e.g., |---|---|) and that all rows maintain consistent column counts
• HTML comment violations: Detects HTML comments inside table cells that could break rendering

The script supports --format json output for CI integration, allowing automated pipelines to parse results programmatically.

Running the Validation Pipeline Locally

Developers can execute the complete validation pipeline locally before submitting pull requests. The scripts are designed to run independently or as a sequence.

Validate all skill definitions and workflow configurations:

python scripts/validate-skills.py

The script exits with code 0 if only warnings are present, or code 1 if any errors are detected.

Validate Markdown syntax across the repository:

python scripts/validate-markdown.py --format json

For CI environments, the JSON output can be piped to tools like jq to fail builds on specific error types:

python scripts/validate-markdown.py --format json | jq 'map(select(.severity == "ERROR")) | length'

Key Files in the Validation System

Understanding the validation architecture requires familiarity with these critical components:

• scripts/validate-skills.py – Central orchestrator for skill-file and workflow validation, containing checker classes like YamlChecker, WorkflowDefinitionChecker, and ManifestDagChecker
• scripts/validate-markdown.py – Dedicated scanner for Markdown structural integrity, implementing validate_file() and table validation helpers
• scripts/update-docs.py – Post-validation utility that generates version.json and updates repository documentation after successful validation
• skills/<skill-name>/SKILL.md – Primary target of YamlChecker and SectionOrderChecker validation
• commands/workflow-manifest.yaml – DAG definition validated by ManifestDagChecker._detect_cycles
• commands/*.yaml – Individual command definitions inspected by WorkflowDefinitionChecker

Summary

The validation pipeline for Claude Code skills enforces quality through three distinct stages:

• Skill-file validation ensures every SKILL.md contains valid YAML front-matter, required metadata fields, proper section ordering, and exactly five Core Workflow steps
• Workflow definition validation verifies that command YAMLs contain required keys, enum values are valid, file paths resolve correctly, and the workflow manifest forms a directed acyclic graph through cycle detection
• Markdown sanity checks prevent rendering errors by validating fenced code block closure and table structure across all documentation

Any ERROR severity finding fails the pipeline, while WARNING items are logged for review. The system is implemented primarily in scripts/validate-skills.py and scripts/validate-markdown.py, enabling both local development checks and automated CI enforcement.

Frequently Asked Questions

What triggers a validation failure in the Claude Code skills pipeline?

A validation failure occurs when any checker reports an issue with ERROR severity. Common triggers include missing required YAML front-matter fields like name or description, invalid regex patterns in skill names, DAG cycles detected in the workflow manifest, unclosed code blocks in Markdown files, or file paths that do not resolve to actual repository files. The pipeline exits with code 1 when errors are present, blocking releases.

How does the pipeline detect circular dependencies in workflows?

The ManifestDagChecker class in scripts/validate-skills.py implements a depth-first search (DFS) algorithm via the _detect_cycles method to analyze the commands/workflow-manifest.yaml file. It traverses the phase dependency graph to detect any cycles, such as intake -> planning -> intake. If a cycle is found, the checker reports a "DAG cycle detected" error, ensuring the workflow remains a directed acyclic graph and preventing infinite execution loops.

Can I run individual validation stages separately?

Yes, the validation scripts are modular. You can run python scripts/validate-skills.py to execute both skill-file and workflow validation stages together. For Markdown validation specifically, use python scripts/validate-markdown.py. While there is no command-line flag to isolate individual stages within validate-skills.py, the script's architecture uses distinct checker classes (such as YamlChecker and WorkflowDefinitionChecker) that could theoretically be invoked separately if you modify the script's entry point.

What is the difference between ERROR and WARNING severity levels?

ERROR severity indicates a critical violation that prevents the skill from functioning correctly or integrating safely with the workflow system. These include syntax errors, missing required fields, or DAG cycles. The pipeline exits with a non-zero status when errors are present. WARNING severity indicates non-critical issues that deviate from best practices but do not block functionality, such as minor formatting inconsistencies or optional metadata suggestions. Warnings are logged for maintainer review but allow the pipeline to exit successfully.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}