# How the CI Validation Workflow Ensures Quality in AI Engineering Lessons

> Discover how the CI validation workflow in ai engineering from scratch audits lessons, syncs stats, and rebuilds site data automatically with GitHub Actions. Ensure quality with every change.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: internals
- Published: 2026-06-13

---

**The CI validation workflow in `rohitg00/ai-engineering-from-scratch` uses a GitHub Actions pipeline defined in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) to automatically audit lesson structure, sync README statistics, and rebuild site data on every push and pull request.**

The `rohitg00/ai-engineering-from-scratch` repository maintains a rigorous CI validation workflow that runs four sequential jobs to guarantee every lesson meets structural and content standards. This pipeline triggers on changes to lesson content, documentation, or build scripts, ensuring the curriculum remains consistent and error-free. By automating validation through GitHub Actions, the project prevents broken links, invalid quiz schemas, and out-of-sync documentation from reaching the main branch.

## The Four-Stage CI Pipeline

The workflow defined in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) orchestrates four distinct jobs that validate different aspects of the curriculum. Each job runs in a specific context—either on every push to `main`, on pull requests, or as a dependent operation.

### Audit Job: Structural Validation

The **audit** job executes on every `push` and `pull_request` to the `main` branch. It runs `python3 scripts/audit_lessons.py` to enforce structural invariants across all lesson directories.

The script performs the following validations:

- **Directory naming**: Matches against `LESION_DIR_RE` pattern
- **Documentation standards**: Ensures [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) exists, exceeds 200 bytes, and contains a top-level H1 heading
- **Code payload**: Verifies the `code/` folder is non-empty
- **Quiz schema**: Validates JSON against `CANONICAL_QUIZ_KEYS`
- **Internal links**: Checks that all Markdown links resolve using `MD_LINK_RE`

If any check fails, the script prints a concise report and exits with code `1`, causing the CI run to fail and blocking the merge.

*Source:* [`audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/audit_lessons.py) lines 65-88, 97-118, 124-148, 150-166.

### README Counts Sync: Automated Repair

The **readme-counts-sync** job runs only on `push` to `main` and depends on the audit job completing successfully. It performs two main operations:

1. Regenerates [`catalog.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/catalog.json) using `python3 scripts/build_catalog.py`
2. Aligns hard-coded counts in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) with the catalog using `python3 scripts/check_readme_counts.py --fix`

If [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) changes during execution, the job configures a bot identity and pushes the commit, retrying up to five times to avoid race conditions.

*Source:* [`check_readme_counts.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/check_readme_counts.py) lines 23-34 (setup), 61-68 (run with `--fix`), 70-97 (commit logic).

### Site Rebuild: Static Data Generation

The **site-rebuild** job depends on `readme-counts-sync` and runs only on pushes to `main`. It executes `node site/build.js` to regenerate [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js), which powers the curriculum website's index. Like the previous job, it commits and pushes any changes automatically.

*Source:* [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) lines 101-114.

### Drift Detection: PR Warning System

The **readme-counts-drift** job runs exclusively on `pull_request` events. It executes `python3 scripts/check_readme_counts.py` **without** the `--fix` flag to detect mismatches between [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and [`catalog.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/catalog.json).

When drift is detected, the job emits a `::warning::` annotation but does not fail the build. This informs contributors that the main branch will automatically heal the counts upon merge, preventing unnecessary blocking of PRs for generated content changes.

*Source:* [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml) lines 49-67.

## Trigger Logic and Path Filtering

The workflow uses path filters to avoid unnecessary runs. According to the source code in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml), the CI triggers only when changes affect:

- `phases/**`
- `scripts/**`
- [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md)
- [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md)
- `glossary/**`
- [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js)

This optimization ensures that unrelated commits—such as updates to documentation outside the curriculum scope—do not consume CI resources.

```yaml
on:
  push:
    branches: [main]
    paths:
      - "phases/**"
      - "scripts/**"
      - "README.md"
      - "ROADMAP.md"
      - "glossary/**"
      - "site/build.js"
  pull_request:
    branches: [main]
    paths:
      # same list as push

```

## Local Validation Commands

Contributors can replicate the CI validation workflow locally using the same scripts executed in the pipeline. This allows early detection of issues before opening pull requests.

Run the structural audit to check all lesson directories:

```bash
python3 scripts/audit_lessons.py

```

Expected output:

```

audit_lessons.py — 428 lesson(s) checked, 0 issue(s)

```

Verify README counts without making changes:

```bash
python3 scripts/check_readme_counts.py

```

Force-fix README counts (equivalent to the CI sync job):

```bash
python3 scripts/check_readme_counts.py --fix

```

These commands mirror the exact steps performed by the GitHub Actions runners, ensuring local development matches CI behavior.

## Summary

The CI validation workflow in `rohitg00/ai-engineering-from-scratch` provides comprehensive automated quality control through four sequential jobs:

- **Audit**: Validates lesson structure, documentation, code presence, and quiz schemas using [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py)
- **README Sync**: Automatically regenerates [`catalog.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/catalog.json) and fixes hard-coded counts in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) on the main branch
- **Site Rebuild**: Updates [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) to keep the website index current with curriculum changes
- **Drift Detection**: Warns on pull requests when README counts differ from the catalog without blocking merges

Together, these jobs ensure every lesson follows required conventions, contains valid payloads, maintains correct internal links, and keeps public documentation synchronized with the source of truth.

## Frequently Asked Questions

### What triggers the CI validation workflow?

The workflow triggers on `push` and `pull_request` events to the `main` branch, but only when files change within specific paths including `phases/**`, `scripts/**`, [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md), and the site builder. This path filtering prevents CI runs for unrelated repository changes.

### How can I run the lesson audit locally?

Execute `python3 scripts/audit_lessons.py` from the repository root. This script iterates through all lesson directories in `phases/`, validates directory naming against `LESION_DIR_RE`, checks that [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) meets minimum size requirements and contains an H1, verifies non-empty `code/` folders, and ensures quiz JSON conforms to `CANONICAL_QUIZ_KEYS`.

### Why does the README drift check not block pull requests?

The **readme-counts-drift** job runs without the `--fix` flag and emits only warnings (`::warning::` annotations) because the counts are automatically generated from [`catalog.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/catalog.json). Blocking the PR would force contributors to manually update generated numbers. Instead, the main branch self-heals through the **readme-counts-sync** job immediately after merge.

### Where is the source of truth for lesson counts?

The [`catalog.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/catalog.json) file serves as the canonical source of truth for lesson statistics. The [`scripts/build_catalog.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/build_catalog.py) script generates this file by scanning the repository structure. Both the README badges and the website data derive their counts from this catalog, ensuring consistency across all documentation surfaces.