How the CI Validation Workflow Ensures Quality in AI Engineering Lessons

The CI validation workflow in rohitg00/ai-engineering-from-scratch uses a GitHub Actions pipeline defined in .github/workflows/curriculum.yml to automatically audit lesson structure, sync README statistics, and rebuild site data on every push and pull request.

The rohitg00/ai-engineering-from-scratch repository maintains a rigorous CI validation workflow that runs four sequential jobs to guarantee every lesson meets structural and content standards. This pipeline triggers on changes to lesson content, documentation, or build scripts, ensuring the curriculum remains consistent and error-free. By automating validation through GitHub Actions, the project prevents broken links, invalid quiz schemas, and out-of-sync documentation from reaching the main branch.

The Four-Stage CI Pipeline

The workflow defined in .github/workflows/curriculum.yml orchestrates four distinct jobs that validate different aspects of the curriculum. Each job runs in a specific context—either on every push to main, on pull requests, or as a dependent operation.

Audit Job: Structural Validation

The audit job executes on every push and pull_request to the main branch. It runs python3 scripts/audit_lessons.py to enforce structural invariants across all lesson directories.

The script performs the following validations:

• Directory naming: Matches against LESION_DIR_RE pattern
• Documentation standards: Ensures docs/en.md exists, exceeds 200 bytes, and contains a top-level H1 heading
• Code payload: Verifies the code/ folder is non-empty
• Quiz schema: Validates JSON against CANONICAL_QUIZ_KEYS
• Internal links: Checks that all Markdown links resolve using MD_LINK_RE

If any check fails, the script prints a concise report and exits with code 1, causing the CI run to fail and blocking the merge.

Source: audit_lessons.py lines 65-88, 97-118, 124-148, 150-166.

README Counts Sync: Automated Repair

The readme-counts-sync job runs only on push to main and depends on the audit job completing successfully. It performs two main operations:

1. Regenerates catalog.json using python3 scripts/build_catalog.py
2. Aligns hard-coded counts in README.md with the catalog using python3 scripts/check_readme_counts.py --fix

If README.md changes during execution, the job configures a bot identity and pushes the commit, retrying up to five times to avoid race conditions.

Source: check_readme_counts.py lines 23-34 (setup), 61-68 (run with --fix), 70-97 (commit logic).

Site Rebuild: Static Data Generation

The site-rebuild job depends on readme-counts-sync and runs only on pushes to main. It executes node site/build.js to regenerate site/data.js, which powers the curriculum website's index. Like the previous job, it commits and pushes any changes automatically.

Source: curriculum.yml lines 101-114.

Drift Detection: PR Warning System

The readme-counts-drift job runs exclusively on pull_request events. It executes python3 scripts/check_readme_counts.py without the --fix flag to detect mismatches between README.md and catalog.json.

When drift is detected, the job emits a ::warning:: annotation but does not fail the build. This informs contributors that the main branch will automatically heal the counts upon merge, preventing unnecessary blocking of PRs for generated content changes.

Source: curriculum.yml lines 49-67.

Trigger Logic and Path Filtering

The workflow uses path filters to avoid unnecessary runs. According to the source code in .github/workflows/curriculum.yml, the CI triggers only when changes affect:

• phases/**
• scripts/**
• README.md
• ROADMAP.md
• glossary/**
• site/build.js

This optimization ensures that unrelated commits—such as updates to documentation outside the curriculum scope—do not consume CI resources.

on:
  push:
    branches: [main]
    paths:
      - "phases/**"
      - "scripts/**"
      - "README.md"
      - "ROADMAP.md"
      - "glossary/**"
      - "site/build.js"
  pull_request:
    branches: [main]
    paths:
      # same list as push

Local Validation Commands

Contributors can replicate the CI validation workflow locally using the same scripts executed in the pipeline. This allows early detection of issues before opening pull requests.

Run the structural audit to check all lesson directories:

python3 scripts/audit_lessons.py

Expected output:

audit_lessons.py — 428 lesson(s) checked, 0 issue(s)

Verify README counts without making changes:

python3 scripts/check_readme_counts.py

Force-fix README counts (equivalent to the CI sync job):

python3 scripts/check_readme_counts.py --fix

These commands mirror the exact steps performed by the GitHub Actions runners, ensuring local development matches CI behavior.

Summary

The CI validation workflow in rohitg00/ai-engineering-from-scratch provides comprehensive automated quality control through four sequential jobs:

• Audit: Validates lesson structure, documentation, code presence, and quiz schemas using scripts/audit_lessons.py
• README Sync: Automatically regenerates catalog.json and fixes hard-coded counts in README.md on the main branch
• Site Rebuild: Updates site/data.js to keep the website index current with curriculum changes
• Drift Detection: Warns on pull requests when README counts differ from the catalog without blocking merges

Together, these jobs ensure every lesson follows required conventions, contains valid payloads, maintains correct internal links, and keeps public documentation synchronized with the source of truth.

Frequently Asked Questions

What triggers the CI validation workflow?

The workflow triggers on push and pull_request events to the main branch, but only when files change within specific paths including phases/**, scripts/**, README.md, and the site builder. This path filtering prevents CI runs for unrelated repository changes.

How can I run the lesson audit locally?

Execute python3 scripts/audit_lessons.py from the repository root. This script iterates through all lesson directories in phases/, validates directory naming against LESION_DIR_RE, checks that docs/en.md meets minimum size requirements and contains an H1, verifies non-empty code/ folders, and ensures quiz JSON conforms to CANONICAL_QUIZ_KEYS.

Why does the README drift check not block pull requests?

The readme-counts-drift job runs without the --fix flag and emits only warnings (::warning:: annotations) because the counts are automatically generated from catalog.json. Blocking the PR would force contributors to manually update generated numbers. Instead, the main branch self-heals through the readme-counts-sync job immediately after merge.

Where is the source of truth for lesson counts?

The catalog.json file serves as the canonical source of truth for lesson statistics. The scripts/build_catalog.py script generates this file by scanning the repository structure. Both the README badges and the website data derive their counts from this catalog, ensuring consistency across all documentation surfaces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}