Implementing Convolutional Neural Networks (CNNs) for Image Classification: A Complete Guide

Convolutional Neural Networks (CNNs) are the foundational architecture for computer vision tasks, and the Microsoft AI-For-Beginners curriculum provides a complete, beginner-friendly implementation pipeline using both TensorFlow/Keras and PyTorch.

Implementing CNNs for image classification requires understanding core architectural components and how they combine into an end-to-end training pipeline. This article walks through the exact approach used in the 07-ConvNets lesson of the microsoft/AI-For-Beginners repository, covering data loading, model definition, training loops, and evaluation with minimal, runnable code examples.

Core CNN Architecture Components

The repository's implementation in lessons/4-ComputerVision/07-ConvNets/ demonstrates eight essential building blocks that appear in every modern CNN.

Convolution and Feature Extraction

The convolution layer learns spatial filters that slide across input images, producing feature maps that detect edges, textures, and increasingly complex patterns.

In tfcv.py, this is implemented as tf.keras.layers.Conv2D. In pytorchcv.py, the equivalent is torch.nn.Conv2d.

Activation Functions

ReLU (Rectified Linear Unit) introduces non-linearity after each convolution, enabling the network to learn complex mappings. This appears as tf.keras.layers.ReLU in TensorFlow and torch.nn.ReLU in PyTorch.

Pooling Layers

Max pooling reduces spatial dimensions while preserving the most salient activations, controlling computational cost and providing translation invariance. The repository uses tf.keras.layers.MaxPooling2D and torch.nn.MaxPool2d.

Batch Normalization

Batch normalization stabilizes training by normalizing layer inputs, reducing internal covariate shift and allowing higher learning rates. Found as tf.keras.layers.BatchNormalization and torch.nn.BatchNorm2d.

Classification Head

The fully-connected (Dense) head flattens feature maps into class predictions. In TensorFlow: tf.keras.layers.Dense. In PyTorch: torch.nn.Linear within the classifier module.

Regularization

Dropout randomly deactivates neurons during training to prevent overfitting. Implemented as tf.keras.layers.Dropout and torch.nn.Dropout with a rate of 0.5 in the repository examples.

Loss and Optimization

The training objective uses categorical cross-entropy loss with the Adam optimizer, configured in both tfcv.py and pytorchcv.py.

Training Infrastructure

Complete training loops with validation and early stopping logic are implemented in ConvNetsTF.ipynb and ConvNetsPyTorch.ipynb.

TensorFlow/Keras Implementation

This minimal example mirrors the full implementation in lessons/4-ComputerVision/07-ConvNets/tfcv.py, using the Oxford-IIIT Pet dataset as specified in the lab materials.

import tensorflow as tf
from tensorflow.keras import layers, models

# 1️⃣ Load and preprocess the pet dataset

(train_ds, val_ds), info = tf.keras.utils.image_dataset_from_directory(
    "data/pet_images",  # folder with subfolders per class

    validation_split=0.2,
    subset="both",
    seed=42,
    image_size=(128, 128),
    batch_size=32,
)

# Normalise pixel values

normalise = layers.Rescaling(1.0 / 255)
train_ds = train_ds.map(lambda x, y: (normalise(x), y))
val_ds   = val_ds.map(lambda x, y: (normalise(x), y))

# 2️⃣ Define a simple CNN

model = models.Sequential([
    layers.Conv2D(32, 3, activation='relu', input_shape=(128, 128, 3)),
    layers.MaxPooling2D(),
    layers.Conv2D(64, 3, activation='relu'),
    layers.MaxPooling2D(),
    layers.Conv2D(128, 3, activation='relu'),
    layers.GlobalAveragePooling2D(),
    layers.Dropout(0.5),
    layers.Dense(info.num_classes, activation='softmax')
])

model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# 3️⃣ Train

model.fit(train_ds, validation_data=val_ds, epochs=10)

# 4️⃣ Evaluate

test_loss, test_acc = model.evaluate(val_ds)
print(f"Validation accuracy: {test_acc:.2%}")

Key design decisions in this architecture:

• Three convolutional blocks with increasing filters (32 → 64 → 128)
• GlobalAveragePooling2D replaces flattening to reduce parameters
• Dropout (0.5) before the final Dense layer for regularization

PyTorch Implementation

This equivalent implementation follows lessons/4-ComputerVision/07-ConvNets/pytorchcv.py, demonstrating the same architectural pattern with PyTorch's explicit training loop.

import torch, torch.nn as nn, torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# 1️⃣ Transform & load the pet dataset

transform = transforms.Compose([
    transforms.Resize((128, 128)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5]*3, std=[0.5]*3)
])

train_set = datasets.ImageFolder('data/pet_images', transform=transform)
train_loader = DataLoader(train_set, batch_size=32, shuffle=True, num_workers=2)

# 2️⃣ Simple CNN definition

class SimpleCNN(nn.Module):
    def __init__(self, n_classes):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 32, 3, padding=1), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(64, 128, 3, padding=1), nn.ReLU(),
            nn.AdaptiveAvgPool2d(1)
        )
        self.classifier = nn.Sequential(
            nn.Dropout(0.5),
            nn.Flatten(),
            nn.Linear(128, n_classes)
        )
    def forward(self, x):
        x = self.features(x)
        return self.classifier(x)

model = SimpleCNN(len(train_set.classes)).to('cuda' if torch.cuda.is_available() else 'cpu')
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=1e-3)

# 3️⃣ Training loop

for epoch in range(10):
    model.train()
    for imgs, lbls in train_loader:
        imgs, lbls = imgs.to(model.device), lbls.to(model.device)
        optimizer.zero_grad()
        outputs = model(imgs)
        loss = criterion(outputs, lbls)
        loss.backward()
        optimizer.step()
    print(f"Epoch {epoch+1} – loss: {loss.item():.4f}")

# 4️⃣ Simple validation (reuse train_loader as a quick proxy)

model.eval()
correct, total = 0, 0
with torch.no_grad():
    for imgs, lbls in train_loader:
        imgs, lbls = imgs.to(model.device), lbls.to(model.device)
        preds = model(imgs).argmax(dim=1)
        correct += (preds == lbls).sum().item()
        total += lbls.size(0)
print(f"Training accuracy: {correct/total:.2%}")

Notable PyTorch-specific patterns:

• Explicit forward() method defining the computation graph
• Manual gradient computation with backward() and step()
• Context manager torch.no_grad() for evaluation efficiency

Classic CNN Architectures

The CNN_Architectures.md file in lessons/4-ComputerVision/07-ConvNets/ explains how modern deep networks extend these primitive blocks:

• VGG-16: Deep stacks of 3×3 convolutions with consistent pooling
• ResNet: Skip connections that enable training of 100+ layer networks
• Inception: Parallel convolutions at multiple scales

Both code examples above can be extended to these architectures by replacing the Sequential definition with pre-built implementations from tf.keras.applications or torchvision.models.

Key Repository Files for CNN Implementation

File	Path	Purpose
README.md	lessons/4-ComputerVision/07-ConvNets/	Lesson overview and learning objectives
CNN_Architectures.md	lessons/4-ComputerVision/07-ConvNets/	Visual guide to VGG, ResNet, and Inception
tfcv.py	lessons/4-ComputerVision/07-ConvNets/	Complete TensorFlow/Keras pipeline
pytorchcv.py	lessons/4-ComputerVision/07-ConvNets/	Complete PyTorch pipeline
ConvNetsTF.ipynb	lessons/4-ComputerVision/07-ConvNets/	Interactive TensorFlow notebook
ConvNetsPyTorch.ipynb	lessons/4-ComputerVision/07-ConvNets/	Interactive PyTorch notebook

Summary

• Convolutional layers with ReLU activations form the feature extraction backbone of any CNN for image classification
• Pooling and batch normalization control spatial dimensions and stabilize training, as implemented in tfcv.py and pytorchcv.py
• Global average pooling followed by dropout and a dense layer provides an efficient classification head
• The Microsoft AI-For-Beginners repository provides complete, runnable implementations in both TensorFlow/Keras and PyTorch
• Same architectural principles scale from simple 3-block CNNs to ResNet-50 and beyond

Frequently Asked Questions

What dataset does the AI-For-Beginners CNN lesson use?

The 07-ConvNets lesson uses the Oxford-IIIT Pet Dataset, containing images of 37 cat and dog breeds. This dataset provides sufficient complexity for learning CNN fundamentals while remaining computationally manageable for beginners.

How do the TensorFlow and PyTorch implementations differ?

The TensorFlow/Keras version in tfcv.py uses a declarative Sequential API with built-in training loops via model.fit(). The PyTorch version in pytorchcv.py requires explicit forward passes, loss computation, backpropagation, and optimizer steps. Both achieve identical architectures but demonstrate different framework philosophies.

Can I replace the simple CNN with ResNet or VGG?

Yes. The repository's CNN_Architectures.md explains how classic architectures are constructed from the same primitives. For immediate experimentation, substitute the Sequential model definition with tf.keras.applications.ResNet50 or torchvision.models.resnet50, adjusting the final layer's output dimensions to match your number of classes.

What image size should I use for CNN classification?

The repository examples use 128×128 pixel inputs for rapid experimentation. For production accuracy, increase to 224×224 (the standard for ImageNet-pretrained models) or your dataset's native resolution, adjusting the architecture's pooling and fully-connected dimensions accordingly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}