CI/CD Workflows in the AI Engineering From Scratch Repository: Automated Curriculum Management

The AI Engineering From Scratch project automates curriculum validation and documentation updates through a single GitHub Actions workflow (.github/workflows/curriculum.yml) that orchestrates four specialized jobs triggered on push and pull request events to the main branch.

The rohitg00/ai-engineering-from-scratch repository maintains a complex curriculum structure that requires continuous validation and synchronization. To manage this without manual intervention, the project implements robust CI/CD workflows defined in the .github/workflows/curriculum.yml file. These automated pipelines ensure lesson consistency, synchronize README statistics, and regenerate the documentation site whenever curriculum changes are proposed or merged.

Workflow Architecture and Event Triggers

The CI/CD implementation uses a single workflow file that responds to two GitHub events with specific path filtering. The workflow triggers on pushes to the main branch and pull requests targeting main, but only when changes occur within relevant paths—including phase directories, scripts, documentation, the site builder, or the workflow definition itself.

This path-gated approach prevents unnecessary execution when unrelated files change, optimizing compute resources while ensuring curriculum integrity.

The Four CI/CD Jobs Explained

The workflow defines four distinct jobs, each handling specific aspects of curriculum maintenance and validation.

Audit: Invariant Checks

The audit job runs on every push and pull request, serving as the primary quality gate. It establishes a fresh Python 3.12 environment and executes scripts/audit_lessons.py to verify lesson consistency across the repository.

This job validates naming conventions, required file presence, and structural invariants before any changes can be merged. If the audit detects inconsistencies, the workflow fails immediately, preventing broken curriculum states from entering the main branch.

README Counts Auto-Fix: Continuous Maintenance

The readme-counts-sync job operates exclusively on pushes to main, automatically correcting documentation drift. It first generates an updated catalog using scripts/build_catalog.py, then executes scripts/check_readme_counts.py --fix to reconcile lesson counts in README.md.

When the script detects mismatches, the job commits the corrected README using bot credentials and pushes the update with built-in retry logic to handle race conditions. This ensures the repository documentation reflects the actual curriculum state without requiring manual bookkeeping.

Site Rebuild: Documentation Generation

The site-rebuild job depends on successful completion of the readme-counts-sync job and runs only on pushes to main. It executes the Node.js-based static site generator via node site/build.js to regenerate site/data.js—the data source powering the project's documentation website.

If the build produces changes to site/data.js, the job commits and pushes the updated file using the same bot identity and retry mechanisms employed in the README synchronization step. This creates a fully automated pipeline where curriculum changes automatically propagate to the published documentation site.

Drift Advisory: Pull Request Validation

The readme-counts-drift job runs exclusively on pull requests, providing early warning of potential synchronization issues. It builds a temporary catalog and executes scripts/check_readme_counts.py (without the --fix flag) to validate current counts against the README.

Rather than auto-fixing, this job emits a warning when it detects drift, alerting maintainers that the main branch will self-heal after merge through the readme-counts-sync job. This advisory approach prevents automated commits to Pull Request branches while ensuring visibility of pending changes.

Local Development and Testing

Developers can replicate the CI/CD checks locally before submitting changes. The following commands mirror the workflow's validation and build processes:

Run the invariant audit locally:

python3 scripts/audit_lessons.py

Manually fix README lesson counts (equivalent to the auto-fix job):

python3 scripts/check_readme_counts.py --fix

Regenerate the site data file for local testing:

node site/build.js

Summary

• The repository centralizes all CI/CD workflows in .github/workflows/curriculum.yml, using path-filtered triggers on main branch events.
• The audit job provides mandatory invariant checking using scripts/audit_lessons.py across Python 3.12 environments.
• The readme-counts-sync job auto-corrects documentation drift by running scripts/check_readme_counts.py --fix and committing changes with bot credentials.
• The site-rebuild job depends on README synchronization and uses node site/build.js to regenerate site/data.js for the documentation site.
• The readme-counts-drift job offers PR-specific validation warnings without modifying branch history.

Frequently Asked Questions

What triggers the CI/CD workflows in the AI Engineering From Scratch repository?

The workflows trigger on push events to the main branch and pull requests targeting main, specifically when changes affect curriculum phases, scripts, documentation, the site builder, or the workflow file itself. This path filtering ensures jobs run only when relevant content changes.

How does the automated README synchronization work?

The readme-counts-sync job generates a fresh catalog via scripts/build_catalog.py, then runs scripts/check_readme_counts.py --fix to identify and correct lesson count discrepancies in README.md. When changes are detected, the job commits and pushes updates using a bot identity with retry logic to prevent race conditions.

Can I run the curriculum validation checks locally?

Yes. Developers can execute python3 scripts/audit_lessons.py to replicate the audit job's invariant checks, and python3 scripts/check_readme_counts.py --fix to preview or apply README corrections. For documentation testing, run node site/build.js to regenerate the site data file locally.

Why does the site-rebuild job depend on readme-counts-sync?

The dependency ensures that the catalog and README counts are fully synchronized before the site generator runs. Since site/build.js relies on accurate curriculum metadata, executing readme-counts-sync first guarantees that the generated site/data.js reflects the most current repository state, preventing documentation staleness.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}