# How the `curriculum.yml` CI Pipeline Prevents Lesson Drift and Ensures Invariant Integrity

> Discover how the curriculum.yml CI pipeline prevents lesson drift and ensures invariant integrity by blocking broken lessons, auto-healing docs, and regenerating the live site.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-08-29

---

**The [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) workflow enforces a defense‑in‑depth strategy that blocks broken lessons pre‑merge, auto‑heals documentation post‑merge, and regenerates the live site to guarantee that the curriculum never drifts from its canonical structure.**

The [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) CI pipeline is the guardian of consistency for the **rohitg00/ai-engineering-from-scratch** repository. It orchestrates a series of invariant checks, automated repairs, and site rebuilds that trigger on every push and pull request affecting lesson content. By treating the codebase as a single source of truth, the pipeline ensures that hundreds of lessons remain synchronized across documentation, certification metadata, and executable artifacts.

## The Defense‑in‑Depth Architecture of the Curriculum CI Pipeline

The pipeline operates across four distinct layers: strict pre‑merge audits, automatic post‑merge repairs, advisory drift detection for contributors, and conflict‑resilient site regeneration. Together, these layers prevent silent corruption of lesson structure, metadata, or presentation.

### Pre‑Merge Invariant Audits

The `audit` job acts as a gatekeeper that runs on every pull request. It executes a battery of Python scripts from the `scripts/` directory to verify that every lesson and certification adheres to the repository’s structural contracts.

- **[`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)** – Scans every lesson directory for required files such as [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md), `code/main.*`, `code/tests/`, and [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json). It enforces the “one‑commit‑per‑lesson” rule documented in the repository’s [`AGENTS.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/AGENTS.md).
- **[`scripts/audit_certifications.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_certifications.py)** – Performs the same validation for certification lessons, ensuring that entry points and test suites exist.
- **`scripts/backfill_certification_references.py --check`** – Verifies that every certification references a concrete lesson, preventing orphaned links in the curriculum graph.
- **Executable labs** – The workflow discovers and runs each certification’s demo ([`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py)) and its unit tests (`test_*.py` files) to guarantee that code ships as runnable artifacts.
- **Skill‑bundle integrity** – [`scripts/test_skill_artifact_bundles.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/test_skill_artifact_bundles.py) ensures that skill bundles do not lose companion files during updates.
- **Bias checks** – `scripts/debias_quizzes.py --check` and `scripts/debias_certification_questions.py --check` validate that answer positions remain unbiased.
- **README translation sync** – `scripts/build_readme_i18n.py --check` confirms that every language version of the README matches the English source.

If any invariant check fails, the CI job aborts immediately, blocking the PR from merging until the violation is remediated.

### Post‑Merge Automated Repairs

Once code lands on the `main` branch, the `readme-counts-sync` job executes to heal documentation drift. The workflow first builds a temporary catalog using [`scripts/build_catalog.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/build_catalog.py), then invokes `scripts/check_readme_counts.py --fix` to rewrite the lesson tables in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and its translation files. This ensures that lesson counts remain accurate without requiring manual edits. After fixing, the workflow commits the changes directly to `main`, preventing stale count drift from persisting.

### Advisory Drift Detection for Contributors

For pull requests, the `readme-counts-drift` job runs the same count script **without** the `--fix` flag. It emits a warning if drift is detected, alerting contributors that the `main` branch will self‑heal after merge. This surfaces issues early while avoiding merge conflicts from automated commits in feature branches.

### Site Regeneration and Live Sync

After the README sync completes on `main`, the `site-rebuild` job executes `node site/build.js` to recreate [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js)—the file the static site consumes for lesson navigation. If the generated file differs from the committed version, the workflow commits and pushes the update, ensuring the live site always reflects the canonical lesson list derived from the source code.

### Conflict‑Resilient Deployment

Both the README sync and site rebuild steps include a retry loop that rebases on the latest `main` and attempts up to five pushes. This logic prevents race conditions between overlapping automation runs from leaving the repository in an inconsistent state.

## Code Examples: Pipeline Configuration and Scripts

The following excerpt from [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) illustrates the `audit` job that enforces invariant checks before any code reaches `main`:

```yaml

# .github/workflows/curriculum.yml

jobs:
  audit:
    name: invariant checks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: run scripts/audit_lessons.py
        run: python3 scripts/audit_lessons.py
      - name: run scripts/audit_certifications.py
        run: python3 scripts/audit_certifications.py
      - name: certification remediation references are complete
        run: python3 scripts/backfill_certification_references.py --check
      - name: run certification lab tests
        run: find certifications/claude/lessons -path '*/code/tests/test_*.py' -print0 | xargs -0 -r -n1 python3
      - name: run certification lab demos
        run: find certifications/claude/lessons -path '*/code/main.py' -print0 | xargs -0 -r -n1 python3

```

The `readme-counts-sync` job demonstrates how the pipeline auto‑heals README drift after a push to `main`:

```yaml
  readme-counts-sync:
    name: README counts auto-fix (main only)
    runs-on: ubuntu-latest
    if: github.event_name == 'push'
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.ref }}
          token: ${{ secrets.GITHUB_TOKEN }}
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: build ephemeral catalog
        run: python3 scripts/build_catalog.py
      - name: sync README counts
        run: python3 scripts/check_readme_counts.py --fix
      - name: regenerate README translations from the synced English
        run: python3 scripts/build_readme_i18n.py

```

## Summary

- **The [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) pipeline triggers on every push and pull request** affecting `phases/**`, `certifications/**`, or validation scripts, ensuring continuous invariant enforcement.
- **Pre‑merge audits** use [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) and related scripts to block PRs that violate structural rules, missing files, or broken tests.
- **Post‑merge automation** repairs README count drift using `scripts/check_readme_counts.py --fix`, keeping human‑readable documentation synchronized with the source of truth.
- **Advisory warnings** on pull requests surface drift to contributors without creating merge conflicts.
- **Site regeneration** via `node site/build.js` guarantees that the public UI consumes an accurate, freshly generated [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js).
- **Retry logic with rebasing** ensures that automated commits succeed even under concurrent modification.

## Frequently Asked Questions

### What triggers the [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) CI pipeline?

The pipeline triggers on both `push` and `pull_request` events for any file that could affect lesson content, including paths under `phases/**`, `certifications/**`, and the `scripts/audit_*.py` validation scripts. This ensures that every structural or content change undergoes invariant verification before merging.

### How does the pipeline handle race conditions during automated commits?

The `readme-counts-sync` and `site-rebuild` jobs implement a retry loop that pulls the latest `main`, rebases the automation’s changes, and attempts the push up to five times. This conflict‑resilient logic prevents transient race conditions from leaving the repository in an inconsistent state.

### What happens if a lesson is missing required files?

The `audit` job executes [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), which scans each lesson directory for mandatory files such as [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md), `code/main.*`, and [`quiz.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/quiz.json). If any required file is absent, the script exits with a non‑zero status, causing the CI job to fail and blocking the pull request from merging until the file is restored.

### Why does the pipeline separate drift detection from drift correction?

The pipeline runs [`scripts/check_readme_counts.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/check_readme_counts.py) **without** the `--fix` flag on pull requests to warn contributors about discrepancies, then runs it **with** `--fix` only after the code reaches `main`. This separation prevents automated commits from cluttering feature branches and causing merge conflicts, while still guaranteeing that the `main` branch remains the single source of truth.