How LLMs Are Taught From Scratch in Phase 10: A Complete End-to-End Curriculum

Phase 10 of the AI Engineering From Scratch curriculum teaches large language model development through 13 sequential steps, starting with byte-pair encoding tokenizers and ending with quantized, production-ready inference pipelines.

The rohitg00/ai-engineering-from-scratch repository structures Phase 10 as a linear "build-it-then-use-it" pathway that demystifies how LLMs are taught from scratch by implementing every component from first principles. Each lesson ships with runnable code/main.py files and outputs/ artifacts, enabling you to execute a complete training pipeline from raw text to an API endpoint.

Foundational Components: Tokenization and Data Processing

Building a BPE Tokenizer From First Principles

The curriculum begins in phases/10-llms-from-scratch/01-tokenizers/ with a ground-up implementation of sub-word tokenization algorithms including BPE, WordPiece, and SentencePiece. You learn how raw text converts to integer IDs through the BytePairEncoder class before moving to phases/10-llms-from-scratch/02-building-a-tokenizer/ to validate vocabulary construction.

from pathlib import Path
from bpe import BytePairEncoder

vocab_path = Path("data/vocab.txt")
corpus_path = Path("data/raw_corpus.txt")
bpe = BytePairEncoder(vocab_path, merges=10000)
bpe.train(corpus_path.read_text())
token_ids = bpe.encode("Hello world!")

Streaming Data Pipelines for Pre-training

In phases/10-llms-from-scratch/03-data-pipelines/code/main.py, you construct efficient data loaders that stream raw corpora, apply on-the-fly tokenization, and handle shuffling and batching. This eliminates memory bottlenecks when processing datasets that exceed RAM capacity.

Model Architecture and Distributed Pre-training

Implementing a 124M Parameter Mini-GPT

The phases/10-llms-from-scratch/04-pre-training-mini-gpt/ lesson introduces a 124-million-parameter transformer with 768-dimensional embeddings, 12 attention heads, and 12 layers. The implementation in code/main.py exposes causal masking and cross-entropy loss calculations used in autoregressive language modeling.

import torch, torch.nn as nn

class MiniGPT(nn.Module):
    def __init__(self, vocab_size, d_model=768, n_head=12, n_layer=12):
        super().__init__()
        self.embed = nn.Embedding(vocab_size, d_model)
        self.layers = nn.ModuleList([
            nn.TransformerEncoderLayer(d_model, n_head) for _ in range(n_layer)
        ])
        self.lm_head = nn.Linear(d_model, vocab_size, bias=False)

    def forward(self, ids):
        x = self.embed(ids)
        for layer in self.layers:
            x = layer(x)
        return self.lm_head(x)

Distributed Scaling With FSDP and DeepSpeed

phases/10-llms-from-scratch/05-scaling-distributed/ teaches multi-GPU parallelism using PyTorch's Fully Sharded Data Parallel (FSDP) and DeepSpeed. You implement parameter sharding and gradient accumulation strategies for multi-node training runs, learning to scale beyond single-device memory limits.

Alignment and Fine-tuning Techniques

Supervised Fine-Tuning (SFT) for Instruction Following

The pretrained Mini-GPT moves to phases/10-llms-from-scratch/06-instruction-tuning-sft/ where you apply Supervised Fine-Tuning on instruction-following datasets. This shifts the model from next-token prediction to conversational response generation.

RLHF and PPO Implementation

phases/10-llms-from-scratch/07-rlhf/ implements the complete Reinforcement Learning from Human Feedback loop. You build a reward model and train it using Proximal Policy Optimization (PPO) to align model outputs with human preferences.

reward_model = nn.Sequential(
    MiniGPT(vocab, d_model=768, n_head=12, n_layer=12),
    nn.Linear(768, 1)
)
optimizer = torch.optim.AdamW(reward_model.parameters(), lr=5e-5)
for batch in reward_dataloader:
    loss = reward_loss_fn(reward_model(batch["prompt"]), batch["human_score"])
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

Direct Preference Optimization (DPO)

phases/10-llms-from-scratch/08-dpo/ introduces Direct Preference Optimization as a simpler alternative to PPO, eliminating the need for a separate reward model while still achieving alignment through preference pairs.

Constitutional AI for Self-Improvement

The self-improvement module in phases/10-llms-from-scratch/09-constitutional-ai-self-improvement/ applies rule-based "constitutions" to iteratively refine model behavior without additional human feedback loops.

Evaluation and Production Optimization

Benchmarking and Safety Metrics

phases/10-llms-from-scratch/10-evaluation/ provides scripts to measure perplexity, instruction correctness, and safety metrics against standard benchmarks. You implement custom evaluation harnesses that extend beyond standard academic leaderboards.

Model Quantization (INT8, GPTQ, AWQ, GGUF)

In phases/10-llms-from-scratch/11-quantization/, you compress the trained model to INT8, GPTQ, AWQ, and GGUF formats. The code/main.py demonstrates post-training quantization that reduces memory footprint by 50-75% with minimal accuracy degradation.

from gptq import GPTQQuantizer
quantizer = GPTQQuantizer(model, bits=8)
quantized_model = quantizer.quantize(calibration_data)

Inference Optimization With KV-Caching and Flash Attention

phases/10-llms-from-scratch/12-inference-optimization/ implements KV-caching, Flash Attention, and async-hogwild serving patterns. These techniques reduce latency in production environments, enabling real-time API responses from your trained model.

End-to-End Integration

The Complete LLM Pipeline

The final lesson in phases/10-llms-from-scratch/13-building-complete-llm-pipeline/ assembles all components into a single reproducible workflow. This pipeline orchestrates training, fine-tuning, evaluation, quantization, and serving, accepting raw text input and exposing a production API endpoint.

Summary

• Phase 10 spans 13 sequential lessons covering the complete LLM lifecycle from tokenization through deployment.
• You implement a 124M-parameter transformer with 768-dimensional embeddings and 12 layers using PyTorch's TransformerEncoderLayer.
• Distributed training leverages FSDP and DeepSpeed for multi-node parameter sharding.
• Alignment covers SFT, RLHF with PPO, DPO, and Constitutional AI approaches.
• Production optimization includes INT8/GPTQ/AWQ/GGUF quantization, KV-caching, and Flash Attention for low-latency inference.
• Every lesson includes runnable code/main.py files located in phases/10-llms-from-scratch/{lesson-number}-{lesson-name}/.

Frequently Asked Questions

What specific transformer architecture is implemented in Phase 10?

Phase 10 implements a 124-million-parameter decoder-only transformer with 768-dimensional embeddings, 12 attention heads, and 12 layers. This architecture uses causal masking and cross-entropy loss for autoregressive language modeling, as defined in phases/10-llms-from-scratch/04-pre-training-mini-gpt/code/main.py.

Which distributed training frameworks are covered for scaling?

The curriculum teaches PyTorch FSDP (Fully Sharded Data Parallel) and DeepSpeed for distributed training. You learn to shard model parameters and gradients across multiple GPUs and nodes, enabling training runs that exceed single-device memory capacities in phases/10-llms-from-scratch/05-scaling-distributed/.

What quantization formats are implemented for model compression?

Phase 10 covers INT8, GPTQ, AWQ, and GGUF quantization schemes. These are implemented in phases/10-llms-from-scratch/11-quantization/code/main.py to compress the 124M parameter model for faster inference on limited hardware with minimal accuracy loss.

Does the curriculum include a complete RLHF implementation?

Yes. phases/10-llms-from-scratch/07-rlhf/ provides a complete Reinforcement Learning from Human Feedback pipeline including reward model architecture, PPO training loops, and loss calculations. Additionally, phases/10-llms-from-scratch/08-dpo/ covers Direct Preference Optimization as a streamlined alternative to traditional RLHF.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}