# How site/build.js Generates site/data.js for the Static AI Engineering Curriculum

> Discover how site build js generates site data js for a static AI engineering curriculum. Learn about markdown parsing, validation, and deterministic object serialization for efficient static site generation.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-09-04

---

**The [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) script in the `rohitg00/ai-engineering-from-scratch` repository parses markdown curriculum files, validates lesson structures, and serializes a deterministic JavaScript object to [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js), enabling a completely static site architecture with cache-busted assets and SEO-optimized meta tags.**

The `rohitg00/ai-engineering-from-scratch` project uses a custom static site generator to transform raw markdown curriculum into a structured data layer. Understanding how [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) generates [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) reveals the deterministic pipeline that powers complex navigation, interactive figures, and search optimization without requiring any server-side runtime.

## The Build Pipeline Architecture

The build process consists of eight deterministic steps implemented as pure functions. According to the source code in [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js), the script orchestrates parsing, validation, and serialization into a single JavaScript module consumed by static HTML pages.

### 1. Loading Repository Constants

At lines 15-20, the script defines canonical paths using `REPO_ROOT`, `README_PATH`, `ROADMAP_PATH`, and `OUTPUT_PATH`. These constants anchor all subsequent file operations to the repository root, ensuring the build remains portable across local development and CI environments.

### 2. Parsing Lesson Status from ROADMAP.md

The `parseRoadmap()` function (lines 104-135) scans the markdown table in [`ROADMAP.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/ROADMAP.md), translating ✅, 🚧, and ⬚ emojis into a structured status map. This produces a nested object mapping `{ phase → { lesson → status } }` that tracks completion states across the curriculum.

### 3. Extracting Phases and Lessons from README.md

At lines 138-210, `parseReadme()` walks the [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) line-by-line to detect phase headers and lesson tables. It extracts lesson names, URLs, types (e.g., "Build", "Theory"), language badges (Python, Node, Rust), and merges the status map from the previous step. Each lesson becomes a structured object with properties `{ name, url, type, lang, status }`.

### 4. Loading Optional Learning Path Manifests

The `parseLearningPaths()` function (lines 310-446) reads JSON files from `learning-paths/`, validates lesson IDs against the canonical curriculum, checks for prerequisite cycles, and builds metadata overlays. This allows the static site to render alternative learning trajectories without duplicating core lesson data.

### 5. Discovering Figure Fences and Provider Mappings

To support interactive visualizations, `discoverUsedFigureIds()` (lines 355-395) scans lesson markdown for ` ```figure` code fences and collects unique figure IDs. Subsequently, `buildFigureProviderManifest()` matches each ID against ordered `figures*.js` providers, generating `providerOrder`, `providerVersions`, and `providersByFigure` mappings embedded in the final payload.

### 6. Building SEO Manifests

The `buildSeoManifests()` function (lines 190-275) orchestrates search optimization by calling `lessonDocumentSeo()` on each lesson's `docs/en.md`. This generates SEO-ready payloads containing `title`, `seoTitle`, `description`, `excerpt`, word counts, and stitched navigation links (`previous`, `next`). The system also disambiguates duplicate titles to ensure unique URL slugs.

### 7. Assembling the Final Site Payload

At lines 2097-2099, the script executes `writeDataFile()` (implicit in the main flow) to combine phases, glossary definitions, certifications, learning paths, and figure manifests into a single JavaScript literal. Additional artifacts include `writeSeoArtifacts()` producing `lesson-seo.json` and `certification-seo.json`, plus `writeFigureManifest()` creating `figure-manifest.js`.

### 8. Versioning Script Tags for Cache Busting

The `assetVersion()` function (lines 1060-1080) computes a content hash for each generated file. The build appends this as a query parameter `?v=<hash>` to all `<script>` tags in HTML pages, guaranteeing browsers fetch fresh assets whenever the curriculum updates.

## The Generated Data Structure

The output file `site/data.js` exports a self-contained module assigned to `window.AIFS`. Static pages load this script and interface with the object directly:

```javascript
// site/data.js - auto-generated
window.AIFS = {
  phases: [
    {
      id: 0,
      name: "Setup & Tooling",
      status: "complete",
      lessons: [
        {
          name: "Dev Environment",
          url: "https://github.com/rohitg00/ai-engineering-from-scratch/tree/main/phases/00-setup-and-tooling/01-dev-environment/",
          type: "Build",
          lang: "Python, Node, Rust",
          status: "complete"
        }
        // ... additional lessons
      ]
    }
    // ... additional phases
  ],
  glossary: { /* term: definition pairs */ },
  certifications: { /* tracks and lesson metadata */ },
  learningPaths: { /* optional path overlays */ },
  figureManifest: { /* figure ID to provider mapping */ }
};

```

This structure enables the static site to render complex navigation trees, filter lessons by language or type, and inject SEO meta tags without server-side processing.

## Running the Build Locally

While the repository's CI workflow executes `site/build.js` automatically on every push, developers can regenerate the data layer locally:

```bash

# From the repository root

node site/build.js

```

This command overwrites `site/data.js`, `site/certification-data.js`, `site/figure-manifest.js`, and auxiliary assets including `sitemap.xml` and `build-meta.js`. The deterministic nature of the script ensures identical outputs across environments when source files remain unchanged.

## Summary

- **`site/build.js`** serves as a deterministic static site generator that parses markdown curriculum into structured JavaScript.
- The **eight-step pipeline** extracts lesson metadata from `README.md`, tracks completion status via `ROADMAP.md`, validates learning paths, and discovers interactive figure dependencies.
- **SEO manifests** are generated per-lesson with navigation links, word counts, and disambiguated titles to optimize for search engines.
- **Content hashing** via `assetVersion()` ensures aggressive cache busting for all generated assets.
- The final **`site/data.js`** exports `window.AIFS`, a self-contained data module consumed by static HTML pages to render the complete curriculum interface.

## Frequently Asked Questions

### What is the purpose of site/data.js?

`site/data.js` acts as the runtime data layer for the static curriculum site. It contains the complete lesson hierarchy, glossary definitions, certification tracks, and figure provider mappings. Static HTML pages load this file via a script tag and read `window.AIFS` to populate navigation menus, lesson cards, and meta tags without requiring a backend database or API calls.

### How does the build script handle curriculum updates?

When markdown source files change, `site/build.js` detects new content during the parsing phase and regenerates the entire `window.AIFS` object. The `assetVersion()` function computes new content hashes, appending unique query strings to script tags. This forces browsers to download the updated `data.js` rather than serving cached versions, ensuring learners always see the latest curriculum status and content.

### Can I run the build script without continuous integration?

Yes. As noted at lines 7-8 in the source, you can run `node site/build.js` directly from the repository root. This executes the full pipeline locally, writing all generated files including `site/data.js`, SEO manifests, and figure mappings to disk. Local execution is useful for testing curriculum changes before committing them to the repository.

### How are interactive figures mapped to lessons?

The system uses two functions: `discoverUsedFigureIds()` scans lesson markdown for ` ```figure` fences to collect figure IDs, then `buildFigureProviderManifest()` matches these IDs against available `figures*.js` providers. The resulting mapping is embedded in `window.AIFS.figureManifest`, allowing the static frontend to dynamically load only the JavaScript providers required for figures present on the current lesson page.