# How the 20-Phase Progression Works in AI Engineering From Scratch: From Math Foundations to Autonomous Systems

> Master AI engineering from scratch with this 20-phase progression. Learn math foundations, algorithms, and build autonomous systems using a strict 'Build It, Use It, Ship It' workflow. Eliminate abstraction gaps and code with c...

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-06-22

---

**The 20-phase progression in AI Engineering From Scratch is a linear, dependency-based curriculum where each phase supplies the mathematical and algorithmic primitives required by the next, enforcing a strict "Build It → Use It → Ship It" workflow that eliminates black-box abstraction gaps.**

The repository `rohitg00/ai-engineering-from-scratch` structures its entire curriculum around a carefully sequenced 20-phase progression that transforms learners from math fundamentals to autonomous system builders. This linear roadmap, visualized in the README's mermaid diagram (lines 58–80), ensures that by the time you encounter complex multi-agent architectures in Phase 14, you have already implemented every underlying primitive—from vector dot products to transformer tokenizers—by hand.

## Overview of the 20-Phase Roadmap

The curriculum spans twenty sequential phases, each producing concrete artifacts that serve as prerequisites for subsequent lessons. The progression begins with environment setup and culminates in capstone projects, with each phase following a predictable pedagogical pattern.

| Phase | Core Focus | Typical Lesson Types | Example Artifacts |
|-------|------------|---------------------|-------------------|
| **0** – Setup & Tooling | Environment provisioning, Docker, Git | Build | `dev-environment/` scripts |
| **1** – Math Foundations | Linear algebra, calculus, probability, optimisation | Learn / Build | Vector utilities ([`vectors.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/vectors.py)) |
| **2** – ML Fundamentals | Classical algorithms (regression, trees, SVM) | Build | Decision-tree trainer |
| **3** – Deep Learning Core | From perceptron to mini-framework, PyTorch intro | Build | Back-prop implementation |
| **4** – Vision | Convolutions, CNNs, diffusion models, VLMs | Build | GAN trainer |
| **5** – NLP | Tokenisation, embeddings, seq2seq, transformers | Build | BPE tokenizer |
| **6** – Speech & Audio | Spectrograms, ASR, TTS, audio codecs | Build | Whisper fine-tune |
| **7** – Transformers Deep Dive | Self-attention, multi-head, encoder-decoder | Build | Full transformer |
| **8** – Generative AI | VAEs, GANs, diffusion, conditioning (LoRA) | Build | Stable Diffusion pipeline |
| **9** – Reinforcement Learning | MDPs, DQN, PPO, RLHF | Build | Reward-model trainer |
| **10** – LLMs from Scratch | Tokenisers, pre-training, scaling, inference | Build | Mini-GPT pre-trainer |
| **11** – LLM Engineering | Prompt engineering, RAG, tool calling | Build | Retrieval-augmented pipeline |
| **12** – Multimodal AI | Vision-language, audio-language, long-video | Build | CLIP pre-trainer |
| **13** – Tools & Protocols | Model Context Protocol (MCP), security, async | Build | MCP server |
| **14** – Agent Engineering | ReAct loops, memory, planning, orchestration | Build | Agent loop implementation |
| **15** – Autonomous Systems | Multi-agent collaboration, swarm, ethics | Learn / Build | Swarm-based task manager |
| **16** – Multi-Agent & Swarms | Hierarchical agents, debate, collaboration | Build | Multi-agent debate |
| **17** – Infrastructure & Production | Deployable runtimes, observability, telemetry | Build | LangGraph state machine |
| **18** – Ethics & Alignment | Safety guards, prompt-injection defenses | Learn | PVE defense |
| **19** – Capstone Projects | End-to-end portfolios, real-repo integration | Build | Full-stack AI product |

## The "Build It → Use It → Ship It" Workflow

Every lesson within the 20-phase progression follows a strict three-stage methodology designed to prevent opaque abstractions. First, you **Build** the algorithm from scratch using only fundamental operations. Next, you **Use** the same logic through production-grade libraries to understand API patterns. Finally, you **Ship** a reusable artifact—whether a prompt file, skill module, agent configuration, or MCP server—to the lesson's `outputs/` directory.

This methodology manifests clearly in [`phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py), where you implement vector operations manually before they power gradient descent calculations in Phase 3.

## Lesson Architecture and File Organization

The repository enforces a consistent folder structure across all twenty phases to maintain navigability and automation. Each lesson resides at `phases/<NN>-<phase-name>/<NN>-<lesson-name>/` and contains three mandatory components:

- [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) – Narrative explanations and learning objectives
- `code/` – Runnable implementations in Python, TypeScript, Rust, or Julia
- `outputs/` – Generated artifacts ready for downstream consumption

The CI pipeline validates this structure through [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py), ensuring every lesson builds and ships correctly before deployment. The [`site/build.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/build.js) aggregator then compiles these into a static documentation site, keeping the curriculum synchronized with the source code.

## Cross-Phase Dependencies and Knowledge Transfer

The 20-phase progression creates tight coupling between phases through explicit code reuse. The linear algebra utilities you write in Phase 1—such as the `dot()` function in [`vectors.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/vectors.py)—reappear in Phase 3's back-propagation implementations and again in Phase 14's memory-vector stores.

When you encounter the Adam optimizer in Phase 3, you already understand gradient descent from Phase 1's optimization chapters. When you build the transformer tokenizer in Phase 5 using [`phases/10-llms-from-scratch/01-tokenizers/code/tokenizer.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/01-tokenizers/code/tokenizer.py), you apply the probability theory from Phase 1. This dependency chain ensures that by Phase 14's [`agent_loop.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/agent_loop.py), you have manually constructed every component of the modern AI stack.

## Practical Implementation Examples

The following snippets demonstrate the "Build It" philosophy at different stages of the progression.

**Phase 1: Mathematical Primitives**

In [`phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py), you implement vector operations without NumPy to understand the underlying computation:

```python
def dot(a, b):
    """Compute the dot product of two equal-length vectors."""
    assert len(a) == len(b), "Vectors must be the same length"
    return sum(x * y for x, y in zip(a, b))

```

This function later imports directly into Phase 3's neural network implementations and Phase 14's retrieval systems, demonstrating the curriculum's reuse philosophy.

**Phase 14: Autonomous Agent Loops**

By Phase 14, you implement the ReAct pattern in [`phases/14-agent-engineering/01-the-agent-loop/code/agent_loop.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/14-agent-engineering/01-the-agent-loop/code/agent_loop.py) without external frameworks:

```python
def run(query, tools):
    history = [user(query)]
    for step in range(MAX_STEPS):
        msg = llm(history)
        if msg.tool_calls:
            for call in msg.tool_calls:
                result = tools[call.name](**call.args)
                history.append(tool_result(call.id, result))
            continue
        return msg.content
    raise StepLimitExceeded

```

This raw implementation teaches the fundamental mechanics before you integrate the Model Context Protocol (MCP) from [`phases/13-tools-and-protocols/06-mcp-fundamentals/code/mcp.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/06-mcp-fundamentals/code/mcp.py) in Phase 13, transitioning to the "Use It" stage with production libraries.

## Key Architectural Benefits

The 20-phase progression provides four distinct advantages over traditional tutorial structures:

- **Incremental Mastery** – By Phase 14, you have written every piece of the modern AI stack from raw math to agent workbenches, eliminating knowledge gaps.
- **Reusability** – Each `outputs/` folder produces self-contained artifacts (prompts, skills, agents) that import cleanly into downstream projects.
- **Transparency** – The "Build It First" requirement forces internal understanding before library abstraction, reducing production bugs from hidden framework assumptions.
- **Extensibility** – Adding new lessons requires only creating a new folder; the CI automatically updates [`README.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/README.md) counts and site data.

## Summary

- The **20-phase progression** in AI Engineering From Scratch organizes learning as a linear, dependency-based curriculum from math fundamentals to autonomous systems.
- Each phase follows a **"Build It → Use It → Ship It"** workflow, requiring manual implementation before library usage.
- The folder structure at `phases/<NN>-<phase-name>/<NN>-<lesson-name>/` enforces consistency with `docs/`, `code/`, and `outputs/` directories.
- Code reuse across phases ensures you understand every optimization hyperparameter and attention mechanism before using production frameworks.
- Key implementation files include [`vectors.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/vectors.py) for mathematical foundations and [`agent_loop.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/agent_loop.py) for autonomous systems, with the Model Context Protocol introduced in Phase 13.

## Frequently Asked Questions

### How long does it take to complete all 20 phases?

The curriculum is designed for self-paced progression, with each phase representing approximately 40–60 hours of focused study including implementation, testing, and artifact shipping. The dependency chain requires sequential completion—Phase 3's deep learning core requires the linear algebra and calculus from Phase 1—so skipping phases breaks the pedagogical model.

### Can I skip directly to the Agent Engineering phase?

Skipping is not recommended because the 20-phase progression enforces strict prerequisites. The [`agent_loop.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/agent_loop.py) implementation in Phase 14 imports vector utilities from Phase 1 and transformer tokenizers from Phase 5. Without completing these foundational phases, you encounter opaque abstractions that contradict the curriculum's transparency goals.

### What programming languages does the curriculum use?

While Python serves as the primary language for most phases, the repository includes runnable implementations in TypeScript, Rust, and Julia within the `code/` directories. The [`scripts/audit_lessons.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/scripts/audit_lessons.py) CI pipeline validates executability across all supported languages, ensuring the "Build It" principle applies regardless of your stack.

### How does the Model Context Protocol (MCP) fit into the progression?

MCP is introduced in Phase 13 as the standardized interface for tool-calling, implemented in [`phases/13-tools-and-protocols/06-mcp-fundamentals/code/mcp.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/06-mcp-fundamentals/code/mcp.py), then integrated into Phase 14's agent implementations. The protocol allows the raw [`agent_loop.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/agent_loop.py) you built to communicate with external tools and data sources, bridging the gap between your custom implementations and production multi-agent systems.