Convolutional Neural Network Architecture in Microsoft AI-For-Beginners: ConvNetsPyTorch.ipynb Explained

The ConvNetsPyTorch.ipynb notebook implements three progressively complex CNN architectures—OneConv, MultiLayerCNN, and LeNet—demonstrating how convolution, pooling, and hierarchical feature extraction work in PyTorch from first principles to production-ready models.

The Microsoft AI-For-Beginners repository provides a hands-on introduction to computer vision through its ConvNetsPyTorch.ipynb notebook, located at lessons/4-ComputerVision/07-ConvNets/. This educational resource builds convolutional neural network intuition by starting with manual filter visualization and culminating in a classic LeNet implementation trained on CIFAR-10. Each architecture introduces essential CNN components that modern deep learning systems still rely on.

Single-Layer CNN: The OneConv Foundation

The simplest architecture, OneConv, demonstrates the bare mechanics of convolution without the complexity of pooling or multiple layers.

Architecture Definition

In ConvNetsPyTorch.ipynb lines 53-65, OneConv is defined as:

class OneConv(nn.Module):
    def __init__(self):
        super(OneConv, self).__init__()
        self.conv = nn.Conv2d(in_channels=1, out_channels=9, kernel_size=(5,5))
        self.flatten = nn.Flatten()
        self.fc = nn.Linear(5184, 10)

    def forward(self, x):
        x = nn.functional.relu(self.conv(x))
        x = self.flatten(x)
        x = nn.functional.log_softmax(self.fc(x), dim=1)
        return x

Key Components Explained

• nn.Conv2d(1, 9, 5) — Creates 9 learnable 5×5 filters operating on single-channel grayscale MNIST images. Each filter slides across the 28×28 input, producing a feature map through weight sharing that dramatically reduces parameters versus fully-connected alternatives.

• Output shape calculation — With kernel_size=5 and no padding, each spatial dimension shrinks from 28 to 24 (28 − 5 + 1 = 24), yielding 9 channels of 24×24 activations: shape [batch, 9, 24, 24].

• Flattening to 5184 units — The 9 × 24 × 24 = 5184 values collapse into a vector for classification, demonstrating how convolution preserves spatial structure until the final decision layer.

The notebook trains this model for 5 epochs on MNIST (lines 80-92), achieving reasonable accuracy despite having only one convolutional layer. Post-training visualization (lines 40-46) displays the nine learned filters, revealing edge and stroke detectors that emerge from gradient descent rather than hand-engineering.

Multi-Layer CNN with Pooling: The MultiLayerCNN Architecture

The second architecture introduces stacked convolutions and max-pooling, the hallmark pattern of modern CNN design.

Architecture Definition

From ConvNetsPyTorch.ipynb lines 5-18:

class MultiLayerCNN(nn.Module):
    def __init__(self):
        super(MultiLayerCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(10, 20, 5)
        self.fc = nn.Linear(320, 10)

    def forward(self, x):
        x = self.pool(nn.functional.relu(self.conv1(x)))
        x = self.pool(nn.functional.relu(self.conv2(x)))
        x = x.view(-1, 320)
        x = nn.functional.log_softmax(self.fc(x), dim=1)
        return x

Progressive Feature Hierarchy

1. First convolution–pooling block: 10 filters of 5×5 produce 24×24 feature maps, then MaxPool2d(2, 2) halves dimensions to 12×12 while retaining strongest local activations.

2. Second convolution–pooling block: 20 filters operate on the downsampled 12×12 representations, producing 8×8 maps that pool to 4×4.

3. Final classification: 20 channels × 4 × 4 = 320 features feed into the output layer.

The pooling layers provide two critical properties: translation invariance (small input shifts don't change the max activation) and computational efficiency (halving resolution after each block). This architecture reaches approximately 98% validation accuracy on MNIST within a few epochs—substantially outperforming the single-layer design despite fewer final parameters.

Classic LeNet Implementation: The LeNet Architecture

The third and most sophisticated architecture adapts Yann LeCun's 1998 LeNet-5 design for color image classification on CIFAR-10.

Architecture Definition

From ConvNetsPyTorch.ipynb lines 84-99:

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.conv3 = nn.Conv2d(16, 120, 5)
        self.flat = nn.Flatten()
        self.fc1 = nn.Linear(120, 64)
        self.fc2 = nn.Linear(64, 10)

    def forward(self, x):
        x = self.pool(nn.functional.relu(self.conv1(x)))
        x = self.pool(nn.functional.relu(self.conv2(x)))
        x = nn.functional.relu(self.conv3(x))
        x = self.flat(x)
        x = nn.functional.relu(self.fc1(x))
        x = self.fc2(x)
        return x

Architectural Advancements Over Earlier Models

• RGB input handling: Conv2d(3, 6, 5) accepts 3-channel color images instead of grayscale.

• Three convolutional stages: The pyramid extends deeper—6 → 16 → 120 channels—with the final 5×5 convolution on 5×5 feature maps producing 120 1×1 "semantic neurons" analogous to fully-connected layers operating on spatially-collapsed representations.

• Two-stage classifier: 120 features compress through ReLU-activated hidden layer (64 units) before final 10-way output.

• Logit outputs for CrossEntropyLoss: Unlike earlier models applying log_softmax internally, LeNet returns raw logits allowing nn.CrossEntropyLoss to combine LogSoftmax and NLLLoss for numerical stability.

The notebook loads CIFAR-10 with normalization (lines 2-12), trains with SGD optimizer (lr=0.001, momentum=0.9), and demonstrates how this classic architecture achieves respectable performance on a substantially harder dataset than MNIST.

Convolutional Filter Visualization: Building Intuition

Before training any network, ConvNetsPyTorch.ipynb includes critical pedagogical code (lines 77-79) visualizing hand-crafted edge detectors:

# Vertical and horizontal Sobel-like filters applied to MNIST

plot_convolution(torch.tensor([[-1.,0.,1.],[-1.,0.,1.],[-1.,0.,1.]]), 'Vertical edge filter')
plot_convolution(torch.tensor([[-1.,-1.,-1.],[0.,0.,0.],[1.,1.,1.]]), 'Horizontal edge filter')

This demonstrates how convolution detects local structure: high responses at intensity discontinuities, near-zero on uniform regions. Post-training visualization of learned OneConv filters confirms that gradient descent automatically discovers similar edge and curve detectors, validating the biological inspiration behind CNN architectures.

Training Infrastructure and Reproducibility

The notebook relies on pytorchcv.py, a helper module co-located in lessons/4-ComputerVision/07-ConvNets/, which provides:

• load_mnist(batch_size) — Standardized MNIST loading with train/test splits
• train(model, trainloader, testloader, ...) — Unified training loop with history tracking
• plot_results(history) — Loss and accuracy visualization
• plot_convolution(kernel, name) — Filter application demo
• display_dataset(dataset, classes) — Sample image grid

For LeNet on CIFAR-10, manual data loading replaces the helper:

transform = torchvision.transforms.Compose([
    torchvision.transforms.ToTensor(),
    torchvision.transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=14, shuffle=True)

Complete Reproduction Code

Execute these snippets sequentially to replicate the notebook's three architectures:

import torch
import torch.nn as nn
import torchvision
from torchinfo import summary

# ---------- OneConv: Single-layer CNN ----------

class OneConv(nn.Module):
    def __init__(self):
        super(OneConv, self).__init__()
        self.conv = nn.Conv2d(1, 9, 5)
        self.flatten = nn.Flatten()
        self.fc = nn.Linear(5184, 10)
    def forward(self, x):
        return nn.functional.log_softmax(self.fc(self.flatten(nn.functional.relu(self.conv(x)))), dim=1)

# ---------- MultiLayerCNN: With pooling ----------

class MultiLayerCNN(nn.Module):
    def __init__(self):
        super(MultiLayerCNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(10, 20, 5)
        self.fc = nn.Linear(320, 10)
    def forward(self, x):
        x = self.pool(nn.functional.relu(self.conv1(x)))
        x = self.pool(nn.functional.relu(self.conv2(x)))
        return nn.functional.log_softmax(self.fc(x.view(-1, 320)), dim=1)

# ---------- LeNet: Three conv layers, RGB input ----------

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.conv3 = nn.Conv2d(16, 120, 5)
        self.fc1 = nn.Linear(120, 64)
        self.fc2 = nn.Linear(64, 10)
    def forward(self, x):
        x = self.pool(nn.functional.relu(self.conv1(x)))
        x = self.pool(nn.functional.relu(self.conv2(x)))
        x = nn.functional.relu(self.conv3(x))
        x = nn.Flatten()(x)
        x = nn.functional.relu(self.fc1(x))
        return self.fc2(x)

# Verify architectures

print("OneConv summary:")
summary(OneConv(), input_size=(1, 1, 28, 28))
print("\nMultiLayerCNN summary:")
summary(MultiLayerCNN(), input_size=(1, 1, 28, 28))
print("\nLeNet summary:")
summary(LeNet(), input_size=(1, 3, 32, 32))

Summary

• OneConv establishes convolution fundamentals: sliding filters, weight sharing, and spatial feature maps feeding classification layers.

• MultiLayerCNN introduces the convolution-pooling pyramid: stacked feature extraction with progressive downsampling for translation invariance and computational efficiency.

• LeNet demonstrates RGB handling, deep hierarchies (three conv layers), and modern loss integration (logits with CrossEntropyLoss)—all patterns still central to ResNet, EfficientNet, and vision transformers.

• The notebook's visualization pipeline (hand-crafted filters → learned filters) builds intuitive understanding of why convolution works.

• All implementations reside in microsoft/AI-For-Beginners/lessons/4-ComputerVision/07-ConvNets/ConvNetsPyTorch.ipynb with supporting utilities in pytorchcv.py.

Frequently Asked Questions

What is the purpose of the OneConv architecture in the notebook?

OneConv serves as a minimal viable CNN that isolates the convolution operation for educational clarity. With a single Conv2d layer producing 9 feature maps from 5×5 filters, it demonstrates how spatial structure is preserved, how output dimensions shrink without padding, and how learned filters automatically detect edges and strokes—foundational concepts before introducing pooling or depth.

Why does MultiLayerCNN use MaxPool2d(2, 2) between convolutions?

Max-pooling with 2×2 windows and stride 2 provides translation invariance by outputting only the maximum activation in each local region, making the network robust to small input shifts. It also halves spatial dimensions after each convolution, reducing subsequent layer parameters and computational cost while allowing deeper stacks to capture larger receptive fields with modest compute.

How does the LeNet implementation differ from the original 1998 paper?

This adaptation modifies three aspects: it accepts 32×32 CIFAR-10 inputs rather than 32×32 MNIST (extended to 28×28 in the paper), uses ReLU activations instead of sigmoid/tanh for faster gradient flow, and outputs raw logits for CrossEntropyLoss rather than applying softmax internally. The core architecture—conv-pool-conv-pool-conv-fc-fc—preserves LeCun's hierarchical feature learning design.

What role does pytorchcv.py play in the notebook?

pytorchcv.py provides curriculum-consistent utilities that abstract data loading, training loops, and visualization so learners focus on architecture rather than boilerplate. Functions like load_mnist() and train() ensure comparable experiments across notebook sections, while plot_convolution() and plot_results() enable immediate visual feedback essential for building CNN intuition.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}