How the curriculum.yml Workflow Orchestrates CI Jobs in ai-engineering-from-scratch

The curriculum.yml workflow orchestrates a self-healing CI pipeline through four specialized jobs that validate lesson integrity, synchronize documentation, and regenerate site indexes automatically upon code changes.

The curriculum.yml workflow serves as the central CI orchestrator for the ai-engineering-from-scratch repository. Defined in .github/workflows/curriculum.yml, this workflow coordinates how the curriculum.yml workflow orchestrates CI jobs across the repository. It triggers on specific GitHub events to ensure that lesson content remains synchronized with generated artifacts.

Workflow Triggers and Event Filtering

The workflow activates on two distinct event types, each constrained by path filtering to prevent unnecessary runs. According to the source code in .github/workflows/curriculum.yml, pushes to the main branch trigger execution when modifying lesson files, scripts, documentation, or the workflow itself (lines 4-15). Similarly, pull requests targeting main initiate checks when the same file paths change (lines 16-27).

This path-specific filtering ensures that unrelated changes—such as updates to CI configuration or external documentation—do not consume runner resources. The event filtering logic guarantees that the heavy CI jobs only execute when curriculum content actually changes.

The Four-Job CI Matrix

The workflow defines four specialized jobs running on ubuntu-latest runners, each designed for a specific validation or synchronization task.

Audit Job: Invariant Checking

The audit job executes repository-wide invariant checks via scripts/audit_lessons.py. This job runs on every push event (lines 32-34) and performs the following steps:

• Checks out the repository code (lines 37-40)
• Sets up Python 3.12 (lines 40-42)
• Executes the audit script to validate lesson structure and dependencies (lines 43-44)

This job acts as the first line of defense, ensuring that all curriculum files adhere to structural requirements before any automated fixes are applied.

readme-counts-sync: Automated Documentation Repair

The readme-counts-sync job automatically repairs lesson count tables in README.md. Triggered exclusively on push events (lines 46-48), this job uses scripts/check_readme_counts.py with the --fix flag to correct discrepancies. The process includes:

1. Checking out the code and setting up Python 3.12 (lines 53-60)
2. Building the internal catalog using scripts/build_catalog.py (lines 60-61)
3. Running the check script with automatic fixing enabled (lines 62-63)
4. Committing and pushing changes back to the repository if modifications are detected (lines 65-98)

This job ensures that curriculum statistics displayed in the README remain accurate without manual intervention.

site-rebuild: Site Index Generation

The site-rebuild job regenerates site/data.js, the data source powering the curriculum website. This job depends on the successful completion of readme-counts-sync (line 105) and only runs on push events (lines 100-106). The execution flow includes:

• Checking out the repository (lines 108-111)
• Executing node site/build.js to generate the site index (lines 112-113)
• Committing and pushing the updated site/data.js if changes exist (lines 115-147)

By declaring needs: readme-counts-sync, the workflow guarantees that the README is already up-to-date before the site data is regenerated.

readme-counts-drift: Pull Request Validation

For pull requests, the workflow executes the readme-counts-drift job instead of the sync jobs. This lightweight check (lines 149-152) verifies whether README.md counts are out-of-sync with the current lesson state without modifying the repository. The process runs scripts/check_readme_counts.py without the --fix flag, reporting warnings if drift is detected while keeping the review process non-intrusive.

Orchestration Mechanisms and Safety Features

The curriculum.yml workflow implements several sophisticated mechanisms to prevent conflicts and ensure reliable CI job orchestration.

Job Dependencies create a linear execution chain where site-rebuild waits for readme-counts-sync to complete (line 105). This sequencing ensures data consistency across generated artifacts.

Permission Scoping restricts write access to only the jobs that require it. The workflow requests contents: read globally (lines 29-31) while escalating to contents: write specifically for jobs that commit changes back to the repository.

Idempotent Bot Commits prevent infinite loops by checking the latest commit message for bot prefixes. Both the README sync and site rebuild jobs abort if the previous commit was already bot-generated (lines 73-76 and 123-125), avoiding circular CI triggers.

Retry Logic handles race conditions on the main branch through exponential backoff. When pushing changes, the jobs attempt up to five retries (lines 83-96 and 132-145), ensuring that temporary conflicts do not fail the entire pipeline.

Running the CI Pipeline Locally and Manually

Triggering the full CI pipeline requires pushing curriculum changes that match the path filters:

git add phases/01-foundations/lesson-01/
git commit -m "feat(phase-01/01): add new lesson"
git push origin main

This push matches the phases/** path filter and automatically runs the audit, readme-counts-sync, and site-rebuild jobs.

To check CI results for a pull request locally:

gh pr checkout 42
git push origin HEAD:refs/heads/feature-branch

When the PR opens, GitHub executes the readme-counts-drift job and surfaces warnings if the README requires updates.

Because the jobs are idempotent, manually re-running them through the GitHub UI will not create duplicate commits if the fixes are already applied:

# In the GitHub UI, go to Actions → curriculum → Re-run jobs

Summary

• The .github/workflows/curriculum.yml file defines a four-job CI matrix that validates and repairs curriculum documentation automatically.
• Event filtering ensures jobs only run when lesson files, scripts, or documentation change, conserving runner resources.
• The audit job validates structural integrity, while readme-counts-sync and site-rebuild maintain automated documentation and site indexes on pushes.
• readme-counts-drift provides lightweight validation for pull requests without modifying the repository.
• Safety mechanisms including job dependencies, idempotent bot commits, and retry logic with exponential backoff prevent race conditions and infinite loops.

Frequently Asked Questions

Why does the site-rebuild job depend on readme-counts-sync?

The site-rebuild job declares needs: readme-counts-sync (line 105) to ensure that README.md lesson counts are accurate before regenerating the site index. This dependency guarantees that the site/data.js file reflects the most current curriculum state, preventing synchronization errors between the documentation and the website.

How does the workflow prevent infinite loops when auto-committing changes?

Both the readme-counts-sync and site-rebuild jobs implement idempotency checks by examining the latest commit message for bot prefixes. If the previous commit was already generated by automation (lines 73-76 and 123-125), the jobs abort to avoid creating circular triggers that would endlessly restart the CI pipeline.

What is the difference between readme-counts-sync and readme-counts-drift?

The readme-counts-sync job runs on push events and automatically fixes README.md lesson counts using the --fix flag, committing changes back to the repository. In contrast, readme-counts-drift runs exclusively on pull requests and performs a read-only check to warn about synchronization issues without modifying files, keeping the review process fast and non-intrusive.

What triggers the curriculum.yml workflow in ai-engineering-from-scratch?

The workflow triggers on two events: pushes to the main branch and pull requests targeting main, but only when these events modify specific paths including lesson files, scripts, or documentation. This path filtering prevents the CI jobs from running on unrelated changes like configuration updates or external assets.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}