# Build CNN from Scratch with Keras: AI for Beginners Complete Guide

> Learn to build a CNN from scratch using Keras with this beginner friendly guide. Stack layers, train your model, and understand AI fundamentals from Microsoft's AI For Beginners.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: getting-started
- Published: 2026-08-01

---

**You can build a convolutional neural network from scratch using TensorFlow Keras by stacking `Conv2D`, `MaxPooling2D`, and `Dense` layers, then training with `model.fit()` on normalized image data.**

The **AI for Beginners** curriculum by Microsoft provides a beginner-friendly pathway to master computer vision fundamentals. Located in the `microsoft/AI-For-Beginners` repository, lesson 4 delivers hands-on Jupyter notebooks that teach you how to build CNNs from the ground up using Keras. This guide extracts the exact implementation details from the official `ConvNetsTF.ipynb` notebook to get you training your first image classifier in minutes.

## Understanding the CNN Architecture

A convolutional neural network processes images through hierarchical feature extraction. According to the source code in `lessons/4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb`, the architecture follows a **feature extractor → classifier** pattern.

### Core Components

- **Input Layer** (`tf.keras.layers.Input`): Defines the tensor shape, typically `(28, 28, 1)` for grayscale MNIST images.
- **Convolutional Layers** (`Conv2D`): Learn spatial filters using learnable kernels. The notebook uses `filters=32, kernel_size=3, activation='relu'` to detect edges and textures.
- **Pooling Layers** (`MaxPooling2D`): Reduce spatial dimensions with `pool_size=2`, providing translation invariance and reducing computation.
- **Flatten**: Converts 3-D feature maps into 1-D vectors before feeding into dense layers.
- **Dense Layers**: Perform high-level reasoning. The hidden layer uses `units=128, activation='relu'`, while the output layer uses `units=10, activation='softmax'` for classification.

This stack creates a **progressive abstraction**: early layers detect edges, middle layers combine them into shapes, and final layers classify the object.

## Building Your First Keras CNN

The repository provides a complete, executable implementation in `lessons/4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb` that requires only TensorFlow and NumPy.

### Loading and Preprocessing Data

Normalization stabilizes training by scaling pixel values to the `[0,1]` range.

```python
import tensorflow as tf
import numpy as np

# Load MNIST dataset

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# Reshape and normalize

x_train = x_train[..., np.newaxis] / 255.0   # shape: (60000, 28, 28, 1)

x_test = x_test[..., np.newaxis] / 255.0

```

The `np.newaxis` operation adds a channel dimension required by Keras convolutional layers.

### Defining the Model Architecture

The Sequential API provides a clean, linear stack of layers as implemented in the official notebook:

```python
from tensorflow.keras import layers, models

model = models.Sequential([
    layers.Input(shape=(28, 28, 1)),
    layers.Conv2D(32, 3, activation='relu'),      # Output: (26, 26, 32)

    layers.MaxPooling2D(2),                       # Output: (13, 13, 32)

    layers.Conv2D(64, 3, activation='relu'),      # Output: (11, 11, 64)

    layers.MaxPooling2D(2),                       # Output: (5, 5, 64)

    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dense(10, activation='softmax')
])

```

**Key architectural decisions** in this code:
- **Small kernels (3×3)**: Capture fine details while keeping parameter counts manageable.
- **Doubling filters**: The second convolution uses 64 filters to capture more complex patterns than the first 32-filter layer.
- **Softmax output**: Produces probability distributions across 10 digit classes.

### Compiling and Training

The notebook uses **Adam optimizer** and **sparse categorical crossentropy** for efficient multi-class classification:

```python
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

history = model.fit(x_train, y_train,
                    epochs=10,
                    batch_size=64,
                    validation_split=0.1,
                    verbose=2)

```

The `validation_split=0.1` parameter automatically reserves 10% of training data for validation, allowing you to monitor overfitting without manual data separation.

## Enhancing the Model

The AI for Beginners curriculum suggests several improvements to boost generalization.

### Adding Dropout Regularization

Insert `Dropout(0.5)` between the flatten and dense layers to prevent overfitting:

```python
model = models.Sequential([
    layers.Input(shape=(28, 28, 1)),
    layers.Conv2D(32, 3, activation='relu'),
    layers.MaxPooling2D(),
    layers.Conv2D(64, 3, activation='relu'),
    layers.MaxPooling2D(),
    layers.Flatten(),
    layers.Dropout(0.5),                          # Randomly zero 50% of inputs

    layers.Dense(128, activation='relu'),
    layers.Dense(10, activation='softmax')
])

```

### Data Augmentation Pipeline

For more robust training on limited datasets, prepend an augmentation layer as shown in the notebook's experiment sections:

```python
data_augmentation = tf.keras.Sequential([
    layers.RandomFlip('horizontal'),
    layers.RandomRotation(0.1),
])

model = models.Sequential([
    layers.Input(shape=(28, 28, 1)),
    data_augmentation,                          # Applied during training only

    layers.Conv2D(32, 3, activation='relu'),
    layers.MaxPooling2D(),
    layers.Conv2D(64, 3, activation='relu'),
    layers.MaxPooling2D(),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dense(10, activation='softmax')
])

```

## Visualizing Training Progress

Understanding convergence requires plotting the training history. The notebook includes matplotlib visualizations to diagnose underfitting or overfitting:

```python
import matplotlib.pyplot as plt

plt.plot(history.history['accuracy'], label='train')
plt.plot(history.history['val_accuracy'], label='val')
plt.title('Training & Validation Accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

```

A healthy training run shows both curves increasing together. If validation accuracy plateaus while training accuracy rises, the model is overfitting and requires regularization.

## Summary

- **Clone the repository** at `microsoft/AI-For-Beginners` and open `lessons/4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb` for the canonical implementation.
- **Normalize pixel values** by dividing by 255.0 before feeding images to the network.
- **Stack Conv2D and MaxPooling2D** layers to extract hierarchical features from raw pixels.
- **Use the Sequential API** for clear, readable model definitions that mirror the mathematical flow.
- **Monitor validation metrics** during `model.fit()` to catch overfitting early and adjust hyperparameters.

## Frequently Asked Questions

### How do I install the required dependencies for the AI for Beginners CNN notebooks?

Install the conda environment defined in [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) at the repository root. This file specifies TensorFlow 2.x, PyTorch, Jupyter, and other computer vision libraries required to execute `ConvNetsTF.ipynb` without import errors.

### What is the difference between the Keras and PyTorch implementations in the repository?

The `ConvNetsTF.ipynb` notebook uses Keras's high-level Sequential API with `model.fit()` for training, while `ConvNetsPyTorch.ipynb` implements the same architecture using `torch.nn.Module` and explicit training loops. Both achieve identical accuracy but Keras abstracts tensor operations for beginners, whereas PyTorch exposes the autograd mechanism.

### Why does the CNN use sparse categorical crossentropy instead of categorical crossentropy?

The MNIST labels in `y_train` are integer indices (0-9) rather than one-hot encoded vectors. `sparse_categorical_crossentropy` computes the loss directly from integer labels, saving memory compared to converting labels to one-hot format required by standard `categorical_crossentropy`.

### How can I improve the test accuracy beyond the baseline 98%?

Add **Dropout layers** after flattening to reduce overfitting, implement **data augmentation** with `RandomFlip` and `RandomRotation` to expand your training set artificially, and experiment with **EarlyStopping** callbacks to halt training when validation loss plateaus. The notebook suggests these as extension exercises for deeper learning.