# How the curriculum.yml CI Workflow Validates Lesson Integrity Across Phases and Certifications

> Discover how the curriculum.yml CI workflow validates lesson integrity across phases and certifications. Learn about automated Python audit scripts ensuring structural and content standards for your AI engineering curriculum.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-09-04

---

**The [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) CI workflow acts as a gatekeeper that enforces strict structural and content standards across every lesson and certification track by orchestrating specialized Python audit scripts.**

The `rohitg00/ai-engineering-from-scratch` repository maintains curriculum quality through a comprehensive continuous integration pipeline defined in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml). When a push or pull request touches lesson-related files, this workflow triggers a nine-step validation process that guarantees every phase lesson and certification module conforms to exacting directory structures, documentation standards, quiz schemas, and executable integrity rules.

## CI Workflow Architecture

The workflow initiates on any change to lesson content, checking out the repository and installing **Python 3.12**—the specific interpreter required by the audit suite. From there, it executes a coordinated sequence of scripts that inspect both the `phases/` hierarchy and the `certifications/claude/` tree, ensuring parity between standard coursework and advanced certification tracks.

## Phase Lesson Validation via audit_lessons.py

The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) script serves as the primary invariant checker for all phase lessons. It recursively walks the `phases/*/` directories and applies three categories of validation:

### Directory Naming and Layout Enforcement

Every lesson folder must match the **`NN-slug`** naming convention, where `NN` represents a zero-padded two-digit number. The script verifies the presence of three required subdirectories:

- `docs/` — containing the [`en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/en.md) documentation file
- `code/` — housing executable examples
- `tests/` — containing validation suites

Missing directories or malformed naming patterns immediately fail the CI job.

### Documentation Integrity Checks

Within each [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file, the validator enforces:

- A mandatory **top-level H1 heading**
- Minimum byte-size thresholds to prevent empty documentation
- Markdown link validation to confirm all referenced files exist within the repository

Broken internal links or missing headers trigger specific error codes (such as **L004**) that block the merge.

### Quiz Schema Validation

The script parses every [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) file to guarantee it follows the canonical schema containing `stage`, `question`, `options`, `correct`, and `explanation` fields. It validates that:

- The `options` array contains between **2 and 6 items**
- The `correct` index falls within the valid range of available options
- Legacy keys (`q`, `choices`, `answer`) generate warnings to prompt migration to the current standard

Schema violations produce error codes like **L008** when option counts fall outside acceptable bounds.

## Certification Track Audits

Certification lessons undergo the same rigorous inspection as phase lessons, with additional certification-specific checks.

### Structural Parity with audit_certifications.py

The [`scripts/audit_certifications.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_certifications.py) file mirrors the lesson auditor, applying identical `NN-slug` naming conventions and subdirectory requirements to the `certifications/claude/lessons/` tree. This ensures that certification content maintains the same organizational standards as foundational coursework.

### Remediation Reference Validation

The `scripts/backfill_certification_references.py --check` command scans certification lessons for missing "remediation reference" entries. This step fails the CI if any certification lesson lacks proper reference links, ensuring learners can always trace back to prerequisite material.

### Live Lab Execution

Unlike phase lessons, certification tracks require executable validation. The workflow runs two `find … | xargs` pipelines that:

1. Execute every [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) demo script
2. Run all `code/tests/test_*.py` unit tests

A non-zero exit code from any lab immediately aborts the workflow, guaranteeing that every certification demo runs without errors before reaching production.

## Skill Artifacts and Static Site Integrity

### Bundle Installation Testing

The [`scripts/test_skill_artifact_bundles.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/test_skill_artifact_bundles.py) script validates that skill-artifact bundles install correctly and maintain companion file integrity. This prevents broken dependencies in hands-on exercises.

### Production Link Resolution

Finally, `node site/build.js` rebuilds the entire static documentation site. This step catches any link-resolution errors that might have passed the initial audits but would fail in the generated HTML, ensuring all validated Markdown links resolve correctly in the published curriculum.

## Local Validation and Debugging

Contributors can run the full validation suite locally before submitting pull requests:

```bash

# Full lesson audit with human-readable output

python3 scripts/audit_lessons.py

# JSON-formatted report for programmatic parsing

python3 scripts/audit_lessons.py --json

```

Typical JSON output includes specific error codes and file paths:

```json
{
  "lessons_checked": 123,
  "issues": [
    {
      "rule": "L004",
      "lesson": "phases/05-transformers/02-attention",
      "file": "phases/05-transformers/02-attention/docs/en.md",
      "message": "docs/en.md missing top-level H1"
    },
    {
      "rule": "L008",
      "lesson": "phases/09-rl/01-bandits",
      "file": "phases/09-rl/01-bandits/quiz.json",
      "message": "question[2] options length must be 2..6 (got 1)"
    }
  ]
}

```

To test certification labs locally:

```bash

# Execute all certification demos (stops on first failure)

find certifications/claude/lessons -path '*/code/main.py' -print0 \
  | xargs -0 -r -n1 python3

# Run a specific certification test file

python3 certifications/claude/lessons/32-architect-professional-system-capstone/code/tests/test_main.py

```

## Summary

- The [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) file orchestrates a nine-step validation pipeline triggered by changes to lesson content.
- [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) enforces `NN-slug` directory naming, required subfolders (`docs`, `code`, `tests`), and validates [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) for H1 headings and link integrity.
- Quiz validation ensures [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files contain 2–6 options with valid correct indices and no legacy key schemas.
- [`scripts/audit_certifications.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_certifications.py) applies identical structural checks to the `certifications/claude/` hierarchy.
- Certification tracks undergo live execution testing via `find … | xargs` pipelines that run [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) demos and `test_*.py` suites.
- Reference completeness is verified by `backfill_certification_references.py --check`.
- The workflow concludes with `node site/build.js` to validate link resolution in the generated static site.

## Frequently Asked Questions

### What naming convention must lesson directories follow?

Every lesson directory must follow the **`NN-slug`** pattern, where `NN` is a zero-padded two-digit number (e.g., `01-introduction-to-ml`). The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) script validates this convention across both `phases/` and `certifications/claude/lessons/` directories.

### How does the workflow verify that quiz questions are properly formatted?

The validator parses [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json) files to ensure they contain the fields `stage`, `question`, `options`, `correct`, and `explanation`. It checks that the `options` array contains between 2 and 6 items and that the `correct` index points to a valid position within that array. Legacy keys like `q`, `choices`, or `answer` trigger warnings.

### Can I run the curriculum validation checks on my local machine?

Yes. Run `python3 scripts/audit_lessons.py` from the repository root to perform a full lesson audit. Use the `--json` flag for machine-parseable output. For certification tracks, you can execute individual labs with `python3` or run the full test suites using the `find … | xargs` patterns found in the CI workflow.

### What happens if a certification lab script fails during CI?

If any [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) demo or `code/tests/test_*.py` unit test exits with a non-zero status, the workflow immediately aborts and marks the job as failed. This prevents broken or non-functional code from merging into the main branch, ensuring all certification labs remain executable for learners.