# How site/build.js Automates the AI Curriculum Website for AI Engineering from Scratch > Discover how site/build.js automates the AI curriculum website from scratch. This script executes a 14-stage pipeline for efficient content generation and SEO asset creation. Learn more. - Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch) - Tags: how-to-guide - Published: 2026-06-16 --- **site/build.js automates the AI curriculum website by executing a 14-stage pipeline that parses Markdown curriculum sources, discovers reusable artifacts, synchronizes repository statistics, and generates SEO assets like data.js, sitemap.xml, and llms.txt.** The rohitg00/ai-engineering-from-scratch repository relies on [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) as its single source of truth for website generation. This Node.js script eliminates manual updates by transforming raw curriculum Markdown into structured runtime data, ensuring the live site always reflects the current repository state without manual HTML edits. ## Curriculum Data Ingestion and Parsing The automation begins by reading and parsing the core curriculum Markdown files to establish the data foundation. ### Parsing ROADMAP.md for Lesson Status The `parseRoadmap()` function (lines 31-60) reads the [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md) file and translates lesson-status emojis into a structured lookup table. It maps ✅ to "complete", 🚧 to "in-progress", and ⬚ to "planned", creating a hierarchical object of `{Phase → {lesson → status}}` that feeds into the curriculum visualization. ### Extracting Phase Structure from README.md The `parseReadme()` function (lines 63-131) extracts phase metadata (id, name, description) and lesson tables (name, type, language, link) from [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md). This function merges the status information derived from the roadmap parser, ensuring the frontend receives a unified view of curriculum structure and completion state. ### Compiling Glossary Definitions The `parseGlossary()` function (lines 71-84) processes [`glossary/terms.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/glossary/terms.md) to generate an array of term objects. Each entry contains the term, what it "says" (definition), and what it "means" (explanation), which powers the site's glossary page search and display features. ## Artifact Discovery and Metadata Enrichment Beyond static Markdown, the script performs deep repository traversal to discover reusable learning materials and enrich lesson data with remote content. ### Walking Output Directories for Reusable Artifacts The `discoverArtifacts()` function (lines 112-177) recursively walks every `phases/*/*/outputs/` directory. It reads front-matter from `skill-*.md`, `prompt-*.md`, and `agent-*.md` files, along with [`mission.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/mission.md) documentation, assembling a flat list of reusable "artifacts" available for student reference. ### Fetching Remote Lesson Summaries The `extractLessonMeta()` function (lines 33-68) processes lessons that contain GitHub URLs. For each valid lesson link, it fetches the corresponding [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) file and programmatically extracts a one-line summary and concatenates all H3 headings into a keywords list. This enriches the curriculum data with searchable metadata without requiring manual summary entry. ## Static Asset Generation and Serialization After parsing and enrichment, the script serializes the collected data into JavaScript modules and SEO-critical XML files. ### Generating site/data.js The output generation block (lines 68-78) serializes the complete curriculum object (`PHASES`), glossary array (`GLOSSARY`), and artifacts collection (`ARTIFACTS`) into [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js). This single JavaScript module serves as the runtime data layer consumed by the frontend, enabling dynamic rendering of the curriculum catalog without a backend database. ### Capturing Build Metadata The `writeBuildMeta()` function (lines 100-117) captures the current Git ref (branch or commit SHA) used during the build. This metadata allows the client-side code to fetch the correct version of lesson documentation and display version information to users. ### Creating SEO sitemap.xml The `writeSitemap()` function (lines 88-107) generates a comprehensive [`sitemap.xml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/sitemap.xml) file covering the home page, catalog, glossary, and every individual lesson URL. This automation ensures search engines discover new lessons immediately upon deployment without manual sitemap updates. ### Producing llms.txt for AI Agents The `writeLlms()` function (lines 113-139) creates a plain-text, link-rich map of the entire curriculum at [`site/llms.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/llms.txt). This file is specifically designed for LLM agents and AI tools to browse the curriculum structure, providing a machine-readable index of all learning resources. ## Repository Synchronization and Statistics The build script maintains consistency between the source repository and the generated website by updating documentation and statistics. ### Computing Curriculum Statistics The statistics computation block (lines 52-66) aggregates totals for phases, lessons, completed lessons, extracted summaries, glossary terms, and discovered artifacts. These metrics drive both the website UI and the [`llms.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/llms.txt) marketing content. ### Syncing README Badges and Traffic Stats The `syncReadme()` function (lines 42-80) updates the lesson count badge in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and injects a live traffic statistics block from [`site/stats.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/stats.json). This ensures the repository documentation displays current metrics automatically. ### Updating Marketing Counts in HTML The `syncCounts()` function (lines 84-94) searches through static HTML files ([`index.html`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/index.html), [`catalog.html`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/catalog.html), etc.) and replaces hard-coded numerical placeholders with current lesson, phase, and artifact totals. This keeps marketing copy accurate without manual editing. ## Executing the Build Pipeline The entire automation is orchestrated by the `build()` function (line 602), which executes all stages sequentially and prints progress logs to the console. The script runs automatically on every push via the GitHub Actions workflow defined in [`curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/curriculum.yml). ### Running the Build Locally Execute the script from the repository root to generate all assets manually: ```bash node site/build.js ``` The console emits step-by-step progress indicators (e.g., "🔍 Parsing README.md…", "✅ Generated site/data.js"). ### Consuming Generated Data in the Frontend The generated [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) exports structured curriculum data for import: ```javascript import { PHASES, GLOSSARY, ARTIFACTS } from '/site/data.js'; console.log('Available phases:', PHASES); console.log('Glossary terms:', GLOSSARY); console.log('Reusable artifacts:', ARTIFACTS); ``` ### Inspecting Generated SEO Files Verify sitemap generation and preview the LLM-friendly curriculum map: ```bash # Preview the first 15 lines of the LLM curriculum map cat site/llms.txt | head -n 15 # List all URLs in the sitemap cat site/sitemap.xml | grep '' ``` ## Summary - **site/build.js** serves as the single automation source for the AI Engineering from Scratch curriculum website, located in the rohitg00/ai-engineering-from-scratch repository. - The script parses [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md), [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md), and [`glossary/terms.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/glossary/terms.md) to construct a unified curriculum data model. - It discovers reusable artifacts by walking `phases/*/*/outputs/` directories and reading front-matter from skill and prompt files. - Generated outputs include [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) (frontend data), [`sitemap.xml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/sitemap.xml) (SEO), and [`llms.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/llms.txt) (AI agent consumption). - Repository synchronization functions keep [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) badges and static HTML marketing counts in perfect sync with actual repository statistics. - The build runs automatically via GitHub Actions or manually via `node site/build.js`. ## Frequently Asked Questions ### What is the primary purpose of site/build.js? site/build.js is the build automation script that transforms static Markdown curriculum files into the dynamic data layer and SEO assets required by the AI Engineering from Scratch website. It eliminates manual website updates by parsing repository content and generating runtime files like [`data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/data.js), [`sitemap.xml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/sitemap.xml), and [`llms.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/llms.txt) on every deployment. ### How does site/build.js handle curriculum updates? The script automatically reflects curriculum changes by re-parsing [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md) during each build execution. When lessons are added, removed, or status emojis are updated in the roadmap, the `parseReadme()` and `parseRoadmap()` functions capture these changes and regenerate the data structures without requiring manual code changes. ### What files does site/build.js generate? The automation generates four primary file types: [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) containing the structured curriculum data (`PHASES`, `GLOSSARY`, `ARTIFACTS`), [`site/sitemap.xml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/sitemap.xml) for search engine optimization, [`site/llms.txt`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/llms.txt) as a plain-text curriculum map for AI agents, and updated inline statistics within [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) and static HTML files. ### Can I run site/build.js outside of GitHub Actions? Yes, site/build.js is a standard Node.js script that executes locally. Run `node site/build.js` from the repository root to generate all assets manually, which is useful for local development, debugging SEO files, or previewing curriculum changes before pushing to production.