# How LLMs Are Taught From Scratch in Phase 10: A Complete End-to-End Curriculum

> Learn how LLMs are taught from scratch in Phase 10 of AI Engineering From Scratch. Master 13 steps from tokenizers to production-ready inference pipelines.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-08-26

---

**Phase 10 of the AI Engineering From Scratch curriculum teaches large language model development through 13 sequential steps, starting with byte-pair encoding tokenizers and ending with quantized, production-ready inference pipelines.**

The `rohitg00/ai-engineering-from-scratch` repository structures Phase 10 as a linear "build-it-then-use-it" pathway that demystifies how LLMs are taught from scratch by implementing every component from first principles. Each lesson ships with runnable [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) files and `outputs/` artifacts, enabling you to execute a complete training pipeline from raw text to an API endpoint.

## Foundational Components: Tokenization and Data Processing

### Building a BPE Tokenizer From First Principles

The curriculum begins in `phases/10-llms-from-scratch/01-tokenizers/` with a ground-up implementation of sub-word tokenization algorithms including BPE, WordPiece, and SentencePiece. You learn how raw text converts to integer IDs through the `BytePairEncoder` class before moving to `phases/10-llms-from-scratch/02-building-a-tokenizer/` to validate vocabulary construction.

```python
from pathlib import Path
from bpe import BytePairEncoder

vocab_path = Path("data/vocab.txt")
corpus_path = Path("data/raw_corpus.txt")
bpe = BytePairEncoder(vocab_path, merges=10000)
bpe.train(corpus_path.read_text())
token_ids = bpe.encode("Hello world!")

```

### Streaming Data Pipelines for Pre-training

In [`phases/10-llms-from-scratch/03-data-pipelines/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/03-data-pipelines/code/main.py), you construct efficient data loaders that stream raw corpora, apply on-the-fly tokenization, and handle shuffling and batching. This eliminates memory bottlenecks when processing datasets that exceed RAM capacity.

## Model Architecture and Distributed Pre-training

### Implementing a 124M Parameter Mini-GPT

The `phases/10-llms-from-scratch/04-pre-training-mini-gpt/` lesson introduces a 124-million-parameter transformer with **768-dimensional embeddings**, **12 attention heads**, and **12 layers**. The implementation in [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) exposes causal masking and cross-entropy loss calculations used in autoregressive language modeling.

```python
import torch, torch.nn as nn

class MiniGPT(nn.Module):
    def __init__(self, vocab_size, d_model=768, n_head=12, n_layer=12):
        super().__init__()
        self.embed = nn.Embedding(vocab_size, d_model)
        self.layers = nn.ModuleList([
            nn.TransformerEncoderLayer(d_model, n_head) for _ in range(n_layer)
        ])
        self.lm_head = nn.Linear(d_model, vocab_size, bias=False)

    def forward(self, ids):
        x = self.embed(ids)
        for layer in self.layers:
            x = layer(x)
        return self.lm_head(x)

```

### Distributed Scaling With FSDP and DeepSpeed

`phases/10-llms-from-scratch/05-scaling-distributed/` teaches multi-GPU parallelism using PyTorch's **Fully Sharded Data Parallel (FSDP)** and **DeepSpeed**. You implement parameter sharding and gradient accumulation strategies for multi-node training runs, learning to scale beyond single-device memory limits.

## Alignment and Fine-tuning Techniques

### Supervised Fine-Tuning (SFT) for Instruction Following

The pretrained Mini-GPT moves to `phases/10-llms-from-scratch/06-instruction-tuning-sft/` where you apply **Supervised Fine-Tuning** on instruction-following datasets. This shifts the model from next-token prediction to conversational response generation.

### RLHF and PPO Implementation

`phases/10-llms-from-scratch/07-rlhf/` implements the complete **Reinforcement Learning from Human Feedback** loop. You build a reward model and train it using **Proximal Policy Optimization (PPO)** to align model outputs with human preferences.

```python
reward_model = nn.Sequential(
    MiniGPT(vocab, d_model=768, n_head=12, n_layer=12),
    nn.Linear(768, 1)
)
optimizer = torch.optim.AdamW(reward_model.parameters(), lr=5e-5)
for batch in reward_dataloader:
    loss = reward_loss_fn(reward_model(batch["prompt"]), batch["human_score"])
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

```

### Direct Preference Optimization (DPO)

`phases/10-llms-from-scratch/08-dpo/` introduces **Direct Preference Optimization** as a simpler alternative to PPO, eliminating the need for a separate reward model while still achieving alignment through preference pairs.

### Constitutional AI for Self-Improvement

The self-improvement module in `phases/10-llms-from-scratch/09-constitutional-ai-self-improvement/` applies rule-based "constitutions" to iteratively refine model behavior without additional human feedback loops.

## Evaluation and Production Optimization

### Benchmarking and Safety Metrics

`phases/10-llms-from-scratch/10-evaluation/` provides scripts to measure **perplexity**, instruction correctness, and safety metrics against standard benchmarks. You implement custom evaluation harnesses that extend beyond standard academic leaderboards.

### Model Quantization (INT8, GPTQ, AWQ, GGUF)

In `phases/10-llms-from-scratch/11-quantization/`, you compress the trained model to **INT8**, **GPTQ**, **AWQ**, and **GGUF** formats. The [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) demonstrates post-training quantization that reduces memory footprint by 50-75% with minimal accuracy degradation.

```python
from gptq import GPTQQuantizer
quantizer = GPTQQuantizer(model, bits=8)
quantized_model = quantizer.quantize(calibration_data)

```

### Inference Optimization With KV-Caching and Flash Attention

`phases/10-llms-from-scratch/12-inference-optimization/` implements **KV-caching**, **Flash Attention**, and async-hogwild serving patterns. These techniques reduce latency in production environments, enabling real-time API responses from your trained model.

## End-to-End Integration

### The Complete LLM Pipeline

The final lesson in `phases/10-llms-from-scratch/13-building-complete-llm-pipeline/` assembles all components into a single reproducible workflow. This pipeline orchestrates training, fine-tuning, evaluation, quantization, and serving, accepting raw text input and exposing a production API endpoint.

## Summary

- **Phase 10** spans 13 sequential lessons covering the complete LLM lifecycle from tokenization through deployment.
- You implement a **124M-parameter transformer** with **768-dimensional embeddings** and **12 layers** using PyTorch's `TransformerEncoderLayer`.
- **Distributed training** leverages FSDP and DeepSpeed for multi-node parameter sharding.
- **Alignment** covers SFT, RLHF with PPO, DPO, and Constitutional AI approaches.
- **Production optimization** includes INT8/GPTQ/AWQ/GGUF quantization, KV-caching, and Flash Attention for low-latency inference.
- Every lesson includes runnable [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) files located in `phases/10-llms-from-scratch/{lesson-number}-{lesson-name}/`.

## Frequently Asked Questions

### What specific transformer architecture is implemented in Phase 10?

Phase 10 implements a **124-million-parameter decoder-only transformer** with 768-dimensional embeddings, 12 attention heads, and 12 layers. This architecture uses causal masking and cross-entropy loss for autoregressive language modeling, as defined in [`phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py).

### Which distributed training frameworks are covered for scaling?

The curriculum teaches **PyTorch FSDP (Fully Sharded Data Parallel)** and **DeepSpeed** for distributed training. You learn to shard model parameters and gradients across multiple GPUs and nodes, enabling training runs that exceed single-device memory capacities in `phases/10-llms-from-scratch/05-scaling-distributed/`.

### What quantization formats are implemented for model compression?

Phase 10 covers **INT8**, **GPTQ**, **AWQ**, and **GGUF** quantization schemes. These are implemented in [`phases/10-llms-from-scratch/11-quantization/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/11-quantization/code/main.py) to compress the 124M parameter model for faster inference on limited hardware with minimal accuracy loss.

### Does the curriculum include a complete RLHF implementation?

Yes. `phases/10-llms-from-scratch/07-rlhf/` provides a complete **Reinforcement Learning from Human Feedback** pipeline including reward model architecture, PPO training loops, and loss calculations. Additionally, `phases/10-llms-from-scratch/08-dpo/` covers **Direct Preference Optimization** as a streamlined alternative to traditional RLHF.