# Understanding the Phase Dependency Graph in AI Engineering from Scratch

> Explore the phase dependency graph in AI Engineering from Scratch. Understand prerequisite chains and deterministic learning for reproducible ML pipelines.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: architecture
- Published: 2026-08-26

---

**The AI Engineering from Scratch curriculum implements a directed acyclic graph (DAG) structure where phases and lessons form prerequisite chains, defined in [`site/roadmap.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/roadmap.js) and visualized through interactive Mermaid diagrams to ensure deterministic learning progression and reproducible ML pipelines.**

The *AI Engineering from Scratch* repository by Rohit G00 organizes its 20-phase curriculum as a formal **phase dependency graph** rather than a linear list. This directed acyclic graph (DAG) structure ensures that every lesson builds upon verified prerequisites, both at the curriculum level and within individual hands-on pipelines. The graph is encoded in JavaScript configuration files, rendered in the interactive roadmap UI, and executed by Python orchestrators that enforce topological ordering.

## Curriculum-Wide Phase Dependencies

### The PREREQS Object in roadmap.js

The backbone of the curriculum dependency graph lives in **[`site/roadmap.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/roadmap.js)**, where a JavaScript object named `PREREQS` enumerates the prerequisite edges between phases. Each key represents a phase slug, while the corresponding array lists direct predecessors that must be completed first.

```javascript
// site/roadmap.js – excerpt
const PREREQS = {
  "01-math-foundations": [],
  "02-nlp-foundations": ["01-math-foundations"],
  "03-transformers-deep-dive": ["02-nlp-foundations"],
  // …
  "10-llms-from-scratch": ["09-llm-engineering"],
  "11-llm-engineering": ["10-llms-from-scratch"],
  // …
};

```

Because the mapping is strictly acyclic, the curriculum can be traversed top-down, guaranteeing that any lesson’s required knowledge is already covered. The UI consumes this `PREREQS` object to render the interactive roadmap, allowing learners to click nodes and illuminate their learning path forward.

### Topological Ordering Guarantees

The **topological sort** of the `PREREQS` graph produces a linear ordering of phases where every phase appears before its dependents. This deterministic ordering prevents circular dependencies and ensures that foundational concepts (like math fundamentals in phase 01) always precede advanced topics (like LLM engineering in phase 10).

## Lesson-Level Pipeline: Phase 10 Example

### The 12-Stage LLM Pipeline Graph

Lesson **13-Building-Complete-LLM-Pipeline** (phase 10) contains a concrete, lesson-level **phase dependency graph** that illustrates how the twelve stages of an LLM pipeline depend on one another. The diagram is authored in Mermaid syntax and lives in [`phases/10-llms-from-scratch/13-building-complete-llm-pipeline/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/13-building-complete-llm-pipeline/docs/en.md) (lines 31-54):

```mermaid
graph TD
    S1["01 Tokenizer vocab"] --> S2["02 Trained tokenizer"]
    S2 --> S3["03 Sharded dataset"]
    S3 --> S4["04 Base model checkpoint"]
    S4 --> S5["05 Scaled training recipe"]
    S5 --> S6["06 SFT checkpoint"]
    S6 --> S7["07 Reward model + PPO policy"]
    S6 --> S8["08 DPO policy"]
    S7 --> S9["09 CAI / GRPO refined policy"]
    S8 --> S9
    S9 --> S10["10 Eval report"]
    S9 --> S11["11 Quantized weights"]
    S11 --> S12["12 Inference server"]
    S10 --> GATE["Ship gate"]
    S12 --> GATE

```

The diagram defines the following **hard dependencies** between pipeline stages:

- **01 Tokenizer vocab** → **02 Trained tokenizer** → **03 Sharded dataset** → **04 Base model checkpoint** → **05 Scaled training recipe** → **06 SFT checkpoint**
- **06 SFT checkpoint** branches to both **07 Reward model + PPO** and **08 DPO policy** (parallel execution paths)
- **07** and **08** converge at **09 CAI / GRPO refined policy**
- **09** feeds both **10 Eval report** and **11 Quantized weights** → **12 Inference server**
- **10** and **12** converge at the final **Ship gate**

### Dependency Constraints and Parallelization

The curriculum explicitly notes that **stages 07 and 08 can run in parallel**, while every other transition represents a strict hard dependency. A change to any early stage (such as the tokenizer vocabulary) invalidates all downstream artifacts, enforcing reproducibility across the entire pipeline. The final **Ship gate** depends on both the evaluation report (stage 10) and the inference server (stage 12), ensuring that deployment only occurs after both validation and serving infrastructure are verified.

## Consuming the Dependency Graph

### Interactive Roadmap Rendering

The curriculum UI aggregates the `PREREQS` graph through **[`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js)**, which combines the phase-level dependencies with per-lesson metadata to produce the interactive roadmap visualization. This JavaScript layer transforms the DAG into an SVG-based navigation interface where learners can trace prerequisite chains visually before committing to a learning path.

### Python Orchestrator Execution

At the lesson level, the Python orchestrator in **[`phases/10-llms-from-scratch/13-building-complete-llm-pipeline/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/13-building-complete-llm-pipeline/code/main.py)** reads the dependency manifest, resolves the topological order, and executes stages only when their inputs are present. The orchestrator skips already-hashed artifacts by comparing SHA-256 checksums, mirroring the high-level DAG behavior at the execution layer. This ensures that expensive training stages are not re-run unnecessarily when upstream artifacts remain unchanged.

## Why DAG Structure Matters for ML Engineering

**Deterministic Re-runs** – Every stage outputs a content hash, and downstream stages verify that hash before proceeding. This prevents "silent" data corruption where stale artifacts might contaminate training results.

**Cost-Tracking** – The orchestrator logs wall-clock time and estimated USD cost per stage. The gate logic can abort the pipeline if a budget threshold is exceeded, allowing ML engineers to treat the curriculum graph as a production-grade cost management tool.

**Rollback Planning** – The curriculum teaches students to classify stages as cheap, medium, or expensive to re-run. The DAG automatically determines the minimal set of downstream stages to invalidate when a failure occurs, minimizing computational waste during iterative development.

## Working with the Dependency Graph

### Extracting Curriculum Order with Kahn's Algorithm

You can programmatically extract the linear phase ordering from [`site/roadmap.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/roadmap.js) using a topological sort implementation. This Node.js script parses the `PREREQS` object and applies Kahn's algorithm to produce a valid learning sequence:

```javascript
// Load the prerequisite mapping and list the full dependency order.
import fs from 'fs';
import path from 'path';

const roadmapPath = path.resolve(
  __dirname,
  '../site/roadmap.js'
);
const src = fs.readFileSync(roadmapPath, 'utf8');
const prereqMatch = src.match(/const PREREQS = (\{[\s\S]*?\});/);
const PREREQS = eval('(' + prereqMatch[1] + ')');

// Simple topological sort (Kahn's algorithm)
function topoSort(graph) {
  const indeg = {};
  Object.keys(graph).forEach(k => indeg[k] = 0);
  Object.values(graph).forEach(vs => vs.forEach(v => indeg[v]++));

  const queue = Object.keys(indeg).filter(k => indeg[k] === 0);
  const order = [];

  while (queue.length) {
    const n = queue.shift();
    order.push(n);
    (graph[n] || []).forEach(m => {
      if (--indeg[m] === 0) queue.push(m);
    });
  }
  return order;
}

console.log('Curriculum order:', topoSort(PREREQS));

```

### Executing Lesson Pipelines with DFS

For lesson-level orchestration, this Python snippet extracts the Mermaid graph from the lesson documentation, builds an adjacency list, performs a depth-first topological sort, and executes stages sequentially:

```python
import yaml, json, subprocess, hashlib, pathlib

# Load the Mermaid graph from the lesson doc

doc_path = pathlib.Path(
    "phases/10-llms-from-scratch/13-building-complete-llm-pipeline/docs/en.md"
)
doc = doc_path.read_text()
graph_block = doc.split("```mermaid")[1].split("```")[0]

# Very small parser: each line "A --> B" becomes an edge

edges = [tuple(l.strip().split(" --> ")) for l in graph_block.splitlines()
         if "-->" in l]

# Build adjacency list

adj = {}
for src, dst in edges:
    adj.setdefault(src.strip(), []).append(dst.strip())

# Topological order (DFS)

visited, order = set(), []

def dfs(node):
    if node in visited:
        return
    visited.add(node)
    for nxt in adj.get(node, []):
        dfs(nxt)
    order.append(node)

for node in adj:
    dfs(node)

order.reverse()
print("Execution order:", order)

# Example: run a stage script if its output hash is missing

for stage in order:
    script = pathlib.Path(f"code/{stage.lower().replace(' ', '_')}.py")
    if script.exists():
        out_hash = subprocess.check_output(["sha256sum", script])
        # In practice compare against manifest; here we just run

        subprocess.run(["python3", script])

```

## Summary

- The **phase dependency graph** in *AI Engineering from Scratch* is defined as a DAG in [`site/roadmap.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/roadmap.js) using the `PREREQS` object, ensuring acyclic prerequisite relationships across 20 curriculum phases.
- Lesson **13-Building-Complete-LLM-Pipeline** demonstrates a 12-stage Mermaid graph where stages 07 and 08 run in parallel, while all other transitions enforce strict hard dependencies leading to a final Ship gate.
- The graph is consumed by both the **[`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js)** frontend for interactive visualization and the **[`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py)** Python orchestrator for deterministic pipeline execution with hash-based artifact validation.
- This DAG structure guarantees **deterministic re-runs**, enables **granular cost tracking**, and supports **intelligent rollback** by invalidating only downstream stages when upstream changes occur.

## Frequently Asked Questions

### What file defines the phase dependency graph in AI Engineering from Scratch?

The global **phase dependency graph** is defined in **[`site/roadmap.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/roadmap.js)** through the `PREREQS` JavaScript object, which maps each phase slug to an array of its direct prerequisite phases. Individual lesson-level graphs (such as the 12-stage LLM pipeline) are defined in Mermaid syntax within each lesson's documentation markdown files.

### How does the curriculum handle parallel lesson execution?

According to the Mermaid diagram in [`phases/10-llms-from-scratch/13-building-complete-llm-pipeline/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/13-building-complete-llm-pipeline/docs/en.md), **stages 07 (Reward model + PPO) and 08 (DPO policy)** can execute in parallel because they share the same parent (stage 06) and have no inter-dependencies. The Python orchestrator detects these parallel branches by analyzing the adjacency list structure and can dispatch them simultaneously while maintaining strict sequential ordering for all other dependent stages.

### What happens if an early stage in the LLM pipeline changes?

If an early stage such as **01 Tokenizer vocab** is modified, the **phase dependency graph** invalidates all downstream artifacts because every subsequent stage (02 through 12) transitively depends on it. The orchestrator in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) detects hash mismatches at the changed stage and automatically triggers re-execution of the minimal set of downstream stages required to maintain pipeline consistency, ensuring no stale data propagates to the final Ship gate.

### Can I query the dependency graph programmatically?

Yes. You can parse the `PREREQS` object from [`site/roadmap.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/roadmap.js) using standard JavaScript evaluation or regex extraction, then apply **Kahn's algorithm** or **depth-first search topological sorting** to generate valid learning paths. For lesson-specific graphs, extract the Mermaid blocks from the markdown documentation in `phases/*/docs/en.md` files and parse the `-->` edges to build adjacency lists for custom orchestration or curriculum planning tools.