How site/build.js Generates site/data.js for the Static AI Engineering Curriculum

The site/build.js script in the rohitg00/ai-engineering-from-scratch repository parses markdown curriculum files, validates lesson structures, and serializes a deterministic JavaScript object to site/data.js, enabling a completely static site architecture with cache-busted assets and SEO-optimized meta tags.

The rohitg00/ai-engineering-from-scratch project uses a custom static site generator to transform raw markdown curriculum into a structured data layer. Understanding how site/build.js generates site/data.js reveals the deterministic pipeline that powers complex navigation, interactive figures, and search optimization without requiring any server-side runtime.

The Build Pipeline Architecture

The build process consists of eight deterministic steps implemented as pure functions. According to the source code in site/build.js, the script orchestrates parsing, validation, and serialization into a single JavaScript module consumed by static HTML pages.

1. Loading Repository Constants

At lines 15-20, the script defines canonical paths using REPO_ROOT, README_PATH, ROADMAP_PATH, and OUTPUT_PATH. These constants anchor all subsequent file operations to the repository root, ensuring the build remains portable across local development and CI environments.

2. Parsing Lesson Status from ROADMAP.md

The parseRoadmap() function (lines 104-135) scans the markdown table in ROADMAP.md, translating ✅, 🚧, and ⬚ emojis into a structured status map. This produces a nested object mapping { phase → { lesson → status } } that tracks completion states across the curriculum.

3. Extracting Phases and Lessons from README.md

At lines 138-210, parseReadme() walks the README.md line-by-line to detect phase headers and lesson tables. It extracts lesson names, URLs, types (e.g., "Build", "Theory"), language badges (Python, Node, Rust), and merges the status map from the previous step. Each lesson becomes a structured object with properties { name, url, type, lang, status }.

4. Loading Optional Learning Path Manifests

The parseLearningPaths() function (lines 310-446) reads JSON files from learning-paths/, validates lesson IDs against the canonical curriculum, checks for prerequisite cycles, and builds metadata overlays. This allows the static site to render alternative learning trajectories without duplicating core lesson data.

5. Discovering Figure Fences and Provider Mappings

To support interactive visualizations, discoverUsedFigureIds() (lines 355-395) scans lesson markdown for ```figure code fences and collects unique figure IDs. Subsequently, buildFigureProviderManifest() matches each ID against ordered figures*.js providers, generating providerOrder, providerVersions, and providersByFigure mappings embedded in the final payload.

6. Building SEO Manifests

The buildSeoManifests() function (lines 190-275) orchestrates search optimization by calling lessonDocumentSeo() on each lesson's docs/en.md. This generates SEO-ready payloads containing title, seoTitle, description, excerpt, word counts, and stitched navigation links (previous, next). The system also disambiguates duplicate titles to ensure unique URL slugs.

7. Assembling the Final Site Payload

At lines 2097-2099, the script executes writeDataFile() (implicit in the main flow) to combine phases, glossary definitions, certifications, learning paths, and figure manifests into a single JavaScript literal. Additional artifacts include writeSeoArtifacts() producing lesson-seo.json and certification-seo.json, plus writeFigureManifest() creating figure-manifest.js.

8. Versioning Script Tags for Cache Busting

The assetVersion() function (lines 1060-1080) computes a content hash for each generated file. The build appends this as a query parameter ?v=<hash> to all <script> tags in HTML pages, guaranteeing browsers fetch fresh assets whenever the curriculum updates.

The Generated Data Structure

The output file site/data.js exports a self-contained module assigned to window.AIFS. Static pages load this script and interface with the object directly:

// site/data.js - auto-generated
window.AIFS = {
  phases: [
    {
      id: 0,
      name: "Setup & Tooling",
      status: "complete",
      lessons: [
        {
          name: "Dev Environment",
          url: "https://github.com/rohitg00/ai-engineering-from-scratch/tree/main/phases/00-setup-and-tooling/01-dev-environment/",
          type: "Build",
          lang: "Python, Node, Rust",
          status: "complete"
        }
        // ... additional lessons
      ]
    }
    // ... additional phases
  ],
  glossary: { /* term: definition pairs */ },
  certifications: { /* tracks and lesson metadata */ },
  learningPaths: { /* optional path overlays */ },
  figureManifest: { /* figure ID to provider mapping */ }
};

This structure enables the static site to render complex navigation trees, filter lessons by language or type, and inject SEO meta tags without server-side processing.

Running the Build Locally

While the repository's CI workflow executes site/build.js automatically on every push, developers can regenerate the data layer locally:

# From the repository root

node site/build.js

This command overwrites site/data.js, site/certification-data.js, site/figure-manifest.js, and auxiliary assets including sitemap.xml and build-meta.js. The deterministic nature of the script ensures identical outputs across environments when source files remain unchanged.

Summary

• site/build.js serves as a deterministic static site generator that parses markdown curriculum into structured JavaScript.
• The eight-step pipeline extracts lesson metadata from README.md, tracks completion status via ROADMAP.md, validates learning paths, and discovers interactive figure dependencies.
• SEO manifests are generated per-lesson with navigation links, word counts, and disambiguated titles to optimize for search engines.
• Content hashing via assetVersion() ensures aggressive cache busting for all generated assets.
• The final site/data.js exports window.AIFS, a self-contained data module consumed by static HTML pages to render the complete curriculum interface.

Frequently Asked Questions

What is the purpose of site/data.js?

site/data.js acts as the runtime data layer for the static curriculum site. It contains the complete lesson hierarchy, glossary definitions, certification tracks, and figure provider mappings. Static HTML pages load this file via a script tag and read window.AIFS to populate navigation menus, lesson cards, and meta tags without requiring a backend database or API calls.

How does the build script handle curriculum updates?

When markdown source files change, site/build.js detects new content during the parsing phase and regenerates the entire window.AIFS object. The assetVersion() function computes new content hashes, appending unique query strings to script tags. This forces browsers to download the updated data.js rather than serving cached versions, ensuring learners always see the latest curriculum status and content.

Can I run the build script without continuous integration?

Yes. As noted at lines 7-8 in the source, you can run node site/build.js directly from the repository root. This executes the full pipeline locally, writing all generated files including site/data.js, SEO manifests, and figure mappings to disk. Local execution is useful for testing curriculum changes before committing them to the repository.

How are interactive figures mapped to lessons?

The system uses two functions: discoverUsedFigureIds() scans lesson markdown for ```figure fences to collect figure IDs, then buildFigureProviderManifest() matches these IDs against available figures*.js providers. The resulting mapping is embedded in window.AIFS.figureManifest, allowing the static frontend to dynamically load only the JavaScript providers required for figures present on the current lesson page.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}