# How the AI Engineering from Scratch Website Is Generated: Node.js Build Pipeline Explained

> Discover how the AI Engineering from Scratch website is generated using a Node.js build pipeline. Learn how Markdown files are compiled into static HTML and data bundles.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-08-30

---

**The AI Engineering from Scratch website is built automatically by a Node.js script ([`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js)) that parses Markdown curriculum files and compiles them into static HTML, JavaScript data bundles, and SEO manifests.**

The **rohitg00/ai-engineering-from-scratch** repository uses a custom static site generator to transform its curriculum Markdown into a fully functional learning platform. Unlike traditional static site generators like Jekyll or Hugo, this project employs a bespoke Node.js build pipeline that processes [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md), [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md), and lesson documentation to produce the AI Engineering from Scratch website. The result is a completely static site served directly from the `site/` folder without any server-side rendering requirements.

## The Build Pipeline Architecture

According to the source code in [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js), the generation process follows a deterministic nine-step pipeline. When a commit lands on the `main` branch, a GitHub Action defined in [`.github/workflows/curriculum.yml`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/.github/workflows/curriculum.yml) executes `node site/build.js`, triggering the following automated workflow.

### Step 1: Ingesting Core Documentation

The build script begins by loading the canonical curriculum sources from the repository root. As implemented in lines 4‑6 of [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js), the script reads:

- [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) — Contains the master lesson table with phases, URLs, and metadata
- [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md) — Tracks implementation status using emoji indicators (✅/🚧/⬚)
- [`glossary/terms.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/glossary/terms.md) — Provides terminology definitions

### Step 2: Parsing Curriculum Structure

Two primary functions extract structured data from the raw Markdown:

**`parseRoadmap()`** (lines 4‑34) scans [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md) to extract the completion status of every phase and lesson, mapping emoji indicators to Boolean or tri-state values.

**`parseReadme()`** (lines 38‑88) walks the Markdown tables in [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) to build a canonical registry of phases and lessons. This includes extracting lesson names, URLs, content types, programming languages, and cross-referencing them against the roadmap status.

### Step 3: Discovering Learning Path Overlays

The **`parseLearningPaths()`** function (lines 7‑45) scans the `learning-paths/` directory for JSON configuration files. These overlays create ordered learning tracks that reference the canonical lessons parsed from [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md), allowing the site to present alternative navigation sequences without duplicating content.

### Step 4: Resolving Interactive Figure Dependencies

The build system maintains a sophisticated figure provider architecture for interactive diagrams:

1. **`discoverUsedFigureIds()`** (lines 35‑45) recursively scans all `*.md` files in `phases/` and `certifications/` directories, searching for `` ```figure `` code fences and extracting unique figure IDs.
2. **`buildFigureProviderManifest()`** (lines 58‑90) matches discovered figure IDs against the ordered list of provider scripts defined in `FIGURE_PROVIDER_ORDER`.
3. The resulting manifest is serialized to `site/figure-manifest.js`, establishing the mapping between figure IDs and their rendering implementations.

### Step 5: Generating SEO Metadata

The **`buildSeoManifests()`** function (lines 19‑28) processes each lesson's `docs/en.md` file to extract:
- Title and summary descriptions
- Keywords and excerpts
- Canonical URLs

It outputs two JSON files:
- `site/lesson-seo.json` — Metadata for individual lessons
- `site/certification-seo.json` — Metadata for certification tracks

### Step 6: Rendering Discovery Interfaces

Using the SEO data structures, the script generates HTML discovery tables:

- **`renderCatalogDiscovery()`** (lines 38‑55) produces HTML rows for the main curriculum catalog.
- **`renderCertificationDiscovery()`** generates rows for the certifications page.

These HTML fragments are inserted into `site/catalog.html` and `site/certifications.html` between special marker comments, replacing previous generated content.

### Step 7: Writing Output Artifacts

The final phase writes all generated assets to disk (lines 24‑33 and 81‑92):

- **`site/data.js`** — A JavaScript module exposing `window.AIFS_DATA`, containing the complete phases/lessons hierarchy, metadata, and figure mappings.
- **`site/figure-manifest.js`** — The figure provider registry.
- **`site/lesson-seo.json`** and **`site/certification-seo.json`** — SEO manifests.
- Updated **`catalog.html`** and **`certifications.html`** with fresh discovery table markup.

## Continuous Integration and Deployment

The repository uses GitHub Actions for automated regeneration. The workflow `curriculum.yml` triggers on every push to `main`, executes `node site/build.js`, then commits the regenerated `site/data.js` (the only generated file versioned in the repository) back to the main branch. This ensures the AI Engineering from Scratch website always reflects the latest curriculum changes without manual intervention.

## Working with the Generated Data

Because the site is purely static, all dynamic behavior relies on the JavaScript data bundles created by the build process.

### Accessing Curriculum Data Client-Side

After the build completes, `site/data.js` creates a global data structure:

```javascript
// Loaded via <script src="data.js"></script>
const phases = window.AIFS_DATA.phases;
const totalLessons = phases.reduce((sum, p) => sum + p.lessons.length, 0);

// Access a specific lesson
const phase0 = phases.find(p => p.id === 0);
const lesson = phase0.lessons.find(l => l.id === 'setup-environment');
console.log(lesson.name, lesson.url);

```

### Resolving Figure Providers

Interactive figures use the generated manifest to load appropriate rendering scripts:

```javascript
// After loading both data.js and figure-manifest.js
const figureId = 'transformer-attention';
const providers = window.AIFS_FIGURE_PROVIDERS[figureId];

// Load the first available provider
if (providers && providers.length > 0) {
  const scriptSrc = providers[0];
  console.log(`Loading figure provider: ${scriptSrc}`);
}

```

### Generating a Sitemap from SEO Manifests

For search engine optimization workflows, you can process the generated JSON server-side:

```javascript
const fs = require('fs');
const seoData = JSON.parse(fs.readFileSync('site/lesson-seo.json', 'utf8'));

const sitemapEntries = Object.values(seoData.lessons).map(lesson => ({
  url: lesson.canonicalUrl,
  lastmod: new Date().toISOString(),
  priority: 0.8
}));

```

## Summary

- The **AI Engineering from Scratch website** is generated by `site/build.js`, a Node.js script that transforms Markdown curriculum files into static assets.
- The build pipeline parses `README.md`, `ROADMAP.md`, and lesson documentation to create a canonical data structure exposed as `window.AIFS_DATA`.
- **Interactive figures** are resolved through `discoverUsedFigureIds()` and `buildFigureProviderManifest()`, which map figure IDs to specific JavaScript provider scripts.
- **SEO metadata** is extracted from lesson frontmatter and rendered into `lesson-seo.json` and `certification-seo.json` for search engine discoverability.
- The `.github/workflows/curriculum.yml` GitHub Action automates regeneration on every commit, ensuring the static site remains synchronized with repository content.
- All output artifacts—including `data.js`, HTML templates, and figure manifests—are pure static files requiring no server-side runtime.

## Frequently Asked Questions

### What triggers the website generation process?

A GitHub Actions workflow defined in `.github/workflows/curriculum.yml` monitors the `main` branch. On every push, the workflow executes `node site/build.js`, regenerating all static assets and committing the updated `site/data.js` back to the repository.

### How does the build system handle interactive diagrams?

The script **`discoverUsedFigureIds()`** scans lesson Markdown for `` ```figure `` code fences to identify required visualizations. It then matches these IDs against provider scripts listed in `FIGURE_PROVIDER_ORDER` via **`buildFigureProviderManifest()`**, outputting the mapping to [`site/figure-manifest.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/figure-manifest.js) for client-side resolution.

### Can I run the website generator locally without CI?

Yes. From the repository root, run `npm install` to install development dependencies, then execute `node site/build.js`. This regenerates [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js), SEO JSON files, and figure manifests without requiring the GitHub Actions environment.

### What is the difference between [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) and the SEO JSON files?

[`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) is a JavaScript module that defines `window.AIFS_DATA`, containing the complete curriculum hierarchy, lesson metadata, and navigation structure required by the site UI. The SEO JSON files ([`lesson-seo.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/lesson-seo.json) and [`certification-seo.json`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/certification-seo.json)) contain search-engine optimization metadata—titles, descriptions, keywords, and canonical URLs—used for generating HTML meta tags and sitemaps.