How the curriculum.yml CI Workflow Validates Lesson Integrity Across Phases and Certifications

The curriculum.yml CI workflow acts as a gatekeeper that enforces strict structural and content standards across every lesson and certification track by orchestrating specialized Python audit scripts.

The rohitg00/ai-engineering-from-scratch repository maintains curriculum quality through a comprehensive continuous integration pipeline defined in .github/workflows/curriculum.yml. When a push or pull request touches lesson-related files, this workflow triggers a nine-step validation process that guarantees every phase lesson and certification module conforms to exacting directory structures, documentation standards, quiz schemas, and executable integrity rules.

CI Workflow Architecture

The workflow initiates on any change to lesson content, checking out the repository and installing Python 3.12—the specific interpreter required by the audit suite. From there, it executes a coordinated sequence of scripts that inspect both the phases/ hierarchy and the certifications/claude/ tree, ensuring parity between standard coursework and advanced certification tracks.

Phase Lesson Validation via audit_lessons.py

The scripts/audit_lessons.py script serves as the primary invariant checker for all phase lessons. It recursively walks the phases/*/ directories and applies three categories of validation:

Directory Naming and Layout Enforcement

Every lesson folder must match the NN-slug naming convention, where NN represents a zero-padded two-digit number. The script verifies the presence of three required subdirectories:

• docs/ — containing the en.md documentation file
• code/ — housing executable examples
• tests/ — containing validation suites

Missing directories or malformed naming patterns immediately fail the CI job.

Documentation Integrity Checks

Within each docs/en.md file, the validator enforces:

• A mandatory top-level H1 heading
• Minimum byte-size thresholds to prevent empty documentation
• Markdown link validation to confirm all referenced files exist within the repository

Broken internal links or missing headers trigger specific error codes (such as L004) that block the merge.

Quiz Schema Validation

The script parses every quiz.json file to guarantee it follows the canonical schema containing stage, question, options, correct, and explanation fields. It validates that:

• The options array contains between 2 and 6 items
• The correct index falls within the valid range of available options
• Legacy keys (q, choices, answer) generate warnings to prompt migration to the current standard

Schema violations produce error codes like L008 when option counts fall outside acceptable bounds.

Certification Track Audits

Certification lessons undergo the same rigorous inspection as phase lessons, with additional certification-specific checks.

Structural Parity with audit_certifications.py

The scripts/audit_certifications.py file mirrors the lesson auditor, applying identical NN-slug naming conventions and subdirectory requirements to the certifications/claude/lessons/ tree. This ensures that certification content maintains the same organizational standards as foundational coursework.

Remediation Reference Validation

The scripts/backfill_certification_references.py --check command scans certification lessons for missing "remediation reference" entries. This step fails the CI if any certification lesson lacks proper reference links, ensuring learners can always trace back to prerequisite material.

Live Lab Execution

Unlike phase lessons, certification tracks require executable validation. The workflow runs two find … | xargs pipelines that:

1. Execute every code/main.py demo script
2. Run all code/tests/test_*.py unit tests

A non-zero exit code from any lab immediately aborts the workflow, guaranteeing that every certification demo runs without errors before reaching production.

Skill Artifacts and Static Site Integrity

Bundle Installation Testing

The scripts/test_skill_artifact_bundles.py script validates that skill-artifact bundles install correctly and maintain companion file integrity. This prevents broken dependencies in hands-on exercises.

Production Link Resolution

Finally, node site/build.js rebuilds the entire static documentation site. This step catches any link-resolution errors that might have passed the initial audits but would fail in the generated HTML, ensuring all validated Markdown links resolve correctly in the published curriculum.

Local Validation and Debugging

Contributors can run the full validation suite locally before submitting pull requests:

# Full lesson audit with human-readable output

python3 scripts/audit_lessons.py

# JSON-formatted report for programmatic parsing

python3 scripts/audit_lessons.py --json

Typical JSON output includes specific error codes and file paths:

{
  "lessons_checked": 123,
  "issues": [
    {
      "rule": "L004",
      "lesson": "phases/05-transformers/02-attention",
      "file": "phases/05-transformers/02-attention/docs/en.md",
      "message": "docs/en.md missing top-level H1"
    },
    {
      "rule": "L008",
      "lesson": "phases/09-rl/01-bandits",
      "file": "phases/09-rl/01-bandits/quiz.json",
      "message": "question[2] options length must be 2..6 (got 1)"
    }
  ]
}

To test certification labs locally:

# Execute all certification demos (stops on first failure)

find certifications/claude/lessons -path '*/code/main.py' -print0 \
  | xargs -0 -r -n1 python3

# Run a specific certification test file

python3 certifications/claude/lessons/32-architect-professional-system-capstone/code/tests/test_main.py

Summary

• The .github/workflows/curriculum.yml file orchestrates a nine-step validation pipeline triggered by changes to lesson content.
• scripts/audit_lessons.py enforces NN-slug directory naming, required subfolders (docs, code, tests), and validates docs/en.md for H1 headings and link integrity.
• Quiz validation ensures quiz.json files contain 2–6 options with valid correct indices and no legacy key schemas.
• scripts/audit_certifications.py applies identical structural checks to the certifications/claude/ hierarchy.
• Certification tracks undergo live execution testing via find … | xargs pipelines that run main.py demos and test_*.py suites.
• Reference completeness is verified by backfill_certification_references.py --check.
• The workflow concludes with node site/build.js to validate link resolution in the generated static site.

Frequently Asked Questions

What naming convention must lesson directories follow?

Every lesson directory must follow the NN-slug pattern, where NN is a zero-padded two-digit number (e.g., 01-introduction-to-ml). The scripts/audit_lessons.py script validates this convention across both phases/ and certifications/claude/lessons/ directories.

How does the workflow verify that quiz questions are properly formatted?

The validator parses quiz.json files to ensure they contain the fields stage, question, options, correct, and explanation. It checks that the options array contains between 2 and 6 items and that the correct index points to a valid position within that array. Legacy keys like q, choices, or answer trigger warnings.

Can I run the curriculum validation checks on my local machine?

Yes. Run python3 scripts/audit_lessons.py from the repository root to perform a full lesson audit. Use the --json flag for machine-parseable output. For certification tracks, you can execute individual labs with python3 or run the full test suites using the find … | xargs patterns found in the CI workflow.

What happens if a certification lab script fails during CI?

If any code/main.py demo or code/tests/test_*.py unit test exits with a non-zero status, the workflow immediately aborts and marks the job as failed. This prevents broken or non-functional code from merging into the main branch, ensuring all certification labs remain executable for learners.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}