How CNNs and GANs Are Taught in the AI Engineering from Scratch Curriculum

The AI Engineering from Scratch curriculum teaches CNNs and GANs through incremental, hands-on PyTorch implementations that bridge mathematical theory to production-ready code, starting with LeNet-5 and 1-D GANs in dedicated Computer Vision and Generative AI phases.

The rohitg00/ai-engineering-from-scratch repository structures deep learning education around reproducible code rather than abstract theory. When exploring how CNNs and GANs are taught in the AI Engineering from Scratch curriculum, you will find a deliberate progression from mathematical foundations to architectural milestones, reinforced by mandatory coding labs that require modifying hyperparameters and evaluating performance metrics.

CNN Instruction: From LeNet-5 to ResNet-18

Mathematical Foundations of Convolution

Lesson 03 in Phase 4 (Computer Vision) anchors CNN understanding in signal processing fundamentals. Learners study the discrete convolution operation, the convolution theorem, and receptive-field calculations before writing a single line of model code. This theoretical grounding ensures students understand why stride and padding choices directly impact feature-map dimensions.

Architectural Milestones and Design Patterns

The curriculum traces the evolution from LeNet-5 through AlexNet, VGG, and ResNet, analyzing how each generation solved specific bottlenecks. Key concepts include the role of increasing depth, the introduction of batch normalization, and the vanishing-gradient solutions enabled by residual connections. Students examine why ResNet's skip connections allow training of networks with 18+ layers while maintaining gradient flow.

Hands-On Implementation in PyTorch

Practical coding begins with a pure-PyTorch implementation of LeNet-5 in phases/04-computer-vision/03-cnns-lenet-to-resnet/code/main.py. The LeNet5 class defines two convolutional blocks followed by three fully connected layers:

import torch.nn as nn
import torch.nn.functional as F

class LeNet5(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 6, kernel_size=5)      # 28→24

        self.conv2 = nn.Conv2d(6, 16, kernel_size=5)     # 12→8

        self.fc1   = nn.Linear(16 * 5 * 5, 120)
        self.fc2   = nn.Linear(120, 84)
        self.fc3   = nn.Linear(84, num_classes)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2)      # 24→12

        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2)      # 8→4

        x = x.view(x.size(0), -1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        return self.fc3(x)

Students then incrementally refactor this base class into a minimal ResNet-18 by adding residual blocks, observing how each architectural change affects training curves and validation accuracy.

GAN Instruction: Generator versus Discriminator Dynamics

The Minimax Game and Loss Mechanics

Lesson 03 in Phase 8 (Generative AI) deconstructs the adversarial training paradigm through the lens of game theory. The lesson explains the original GAN formulation's minimax objective, contrasting the generator's cross-entropy loss against the discriminator's binary classification task. Common failure modes like mode collapse and vanishing gradients are diagnosed using mathematical proofs before students encounter them empirically.

Evolutionary Improvements and Modern Hybrids

The curriculum maps the progression from vanilla GANs to DCGAN, WGAN-GP, and StyleGAN 1-3, emphasizing how architectural constraints (such as the PatchGAN discriminator) and loss-function improvements (Wasserstein distance) stabilize training. A contemporary angle shows why modern pipelines frequently combine GAN discriminators with diffusion models to compute perceptual loss, reflecting 2024-2025 production practices.

Lightweight 1-D GAN Implementation

The practical component resides in phases/08-generative-ai/03-gans-generator-discriminator/code/main.py, featuring a minimal 1-D GAN that clarifies the adversarial loop without image-complexity overhead. The implementation defines a Generator and a PatchDiscriminator:

import torch
import torch.nn as nn

class Generator(nn.Module):
    def __init__(self, nz=10, hidden=64):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(nz, hidden),
            nn.ReLU(),
            nn.Linear(hidden, hidden),
            nn.ReLU(),
            nn.Linear(hidden, 1)      # output a scalar

        )
    def forward(self, z):
        return self.net(z)

class PatchDiscriminator(nn.Module):
    def __init__(self, hidden=64):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(1, hidden),
            nn.LeakyReLU(0.2),
            nn.Linear(hidden, hidden),
            nn.LeakyReLU(0.2),
            nn.Linear(hidden, 1)      # single “real/fake” logit

        )
    def forward(self, x):
        return self.net(x)

Learners train these modules on synthetic datasets, experimenting with Wasserstein loss variants and learning-rate ratios between networks to observe convergence behavior.

Pedagogical Methodology: Incremental Experimentation

Both lessons enforce an incremental experimentation protocol. Students must run baseline models, then modify a single hyperparameter—such as stride in the CNN or the discriminator-to-generator update ratio in the GAN—before plotting learning curves. Evaluation uses standard metrics: accuracy for CNNs and FID/Inception Score for GANs. This constraint-based approach ensures learners correlate specific code changes with measurable performance shifts, preventing "cargo-cult" deep learning.

Summary

• CNNs are taught via phases/04-computer-vision/03-cnns-lenet-to-resnet/, progressing from convolution mathematics to a working ResNet-18.
• GANs are taught via phases/08-generative-ai/03-gans-generator-discriminator/, covering minimax theory through StyleGAN evolution and 1-D adversarial training.
• Code-first validation requires students to implement LeNet5 and Generator/PatchDiscriminator classes in pure PyTorch.
• Incremental experimentation mandates hyperparameter modification and metric tracking (accuracy, FID) to solidify architectural intuitions.

Frequently Asked Questions

What file paths contain the CNN and GAN lesson documentation?

The CNN lesson documentation lives at phases/04-computer-vision/03-cnns-lenet-to-resnet/docs/en.md with source code in phases/04-computer-vision/03-cnns-lenet-to-resnet/code/main.py. The GAN lesson documentation is located at phases/08-generative-ai/03-gans-generator-discriminator/docs/en.md with its implementation in phases/08-generative-ai/03-gans-generator-discriminator/code/main.py.

Does the curriculum explain why GANs suffer from mode collapse?

Yes, the GAN lesson explicitly covers mode collapse and vanishing gradients as mathematical failure modes of the minimax game before students implement the PatchDiscriminator, ensuring theoretical awareness precedes empirical debugging.

Are the PyTorch implementations suitable for production use?

While the LeNet5, Generator, and PatchDiscriminator classes are intentionally minimal for pedagogical clarity, they follow PyTorch best practices and serve as reference implementations that learners can extend into production-grade architectures.

How does the curriculum connect CNNs to modern computer vision?

The CNN lesson bridges historical architectures (LeNet-5, AlexNet) to ResNet-18 and discusses how residual connections solve vanishing gradients, directly preparing students for transfer learning with contemporary vision transformers and ResNet backbones.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}