Neural Networks Lessons 03-05: Core Topics in Microsoft AI For Beginners

The Neural Networks section (Lessons 03-05) in the microsoft/AI-For-Beginners curriculum covers the Perceptron, multi-layered networks, PyTorch and Keras/TensorFlow frameworks, and overfitting mitigation techniques.

This section of the open-source microsoft/AI-For-Beginners repository transitions learners from theoretical machine learning concepts to practical deep learning implementation. The content is structured within the lessons/3-NeuralNetworks/ directory and progresses through three sequential sub-lessons (03-Perceptron, 04-OwnFramework, and 05-Frameworks) that build foundational skills before introducing industry-standard tools.

Core Topics Covered in the Neural Networks Section

Perceptron

The curriculum begins with the Perceptron, the simplest form of artificial neural network used for binary classification. This topic introduces the mathematical model of a neuron, including weighted inputs, bias, and activation functions, implemented from scratch to demonstrate how single-layer networks learn linear decision boundaries.

According to the Lesson 3 README at line 45, the Perceptron lesson includes a detailed explanation and accompanying notebook that demonstrates training convergence and limitations when handling non-linearly separable data.

Multi-Layered Networks

Building upon the Perceptron, the section advances to multi-layered networks (multi-layer perceptrons) capable of learning complex, non-linear patterns. The line 46 reference indicates this topic includes building a custom neural network framework from scratch, implementing forward propagation, backpropagation, and gradient descent without relying on external libraries.

The accompanying notebook in lessons/3-NeuralNetworks/04-OwnFramework/ demonstrates how to construct hidden layers, initialize weights, and compute derivatives manually—providing essential intuition before abstraction via high-level frameworks.

Neural Network Frameworks

The curriculum introduces two dominant deep learning ecosystems as documented at line 47:

• PyTorch – Covered in 05-Frameworks/IntroPyTorch.ipynb, demonstrating dynamic computation graphs, automatic differentiation, and GPU acceleration.
• Keras/TensorFlow – Covered in 05-Frameworks/IntroKerasTF.ipynb, showcasing the high-level API for rapid model prototyping and deployment.

Both notebooks implement identical architectures to facilitate direct comparison between imperative (PyTorch) and declarative (Keras) programming styles.

Overfitting

The line 48 reference identifies overfitting as a critical topic addressed within the framework notebooks. The curriculum explains detection methods (validation loss monitoring) and mitigation strategies including early stopping, dropout layers, and L2 regularization to improve model generalization.

Practical Implementation Examples

The following code snippets illustrate the progression from scratch implementations to framework-based solutions:

# Perceptron implementation from scratch

import numpy as np

def perceptron(X, y, lr=0.1, epochs=10):
    w = np.zeros(X.shape[1])
    b = 0
    for _ in range(epochs):
        for xi, target in zip(X, y):
            pred = np.sign(np.dot(xi, w) + b)
            update = lr * (target - pred)
            w += update * xi
            b += update
    return w, b

# Multi-layered network manual feed-forward

def sigmoid(z): 
    return 1 / (1 + np.exp(-z))

def forward(x, W1, b1, W2, b2):
    z1 = np.dot(x, W1) + b1
    a1 = sigmoid(z1)
    z2 = np.dot(a1, W2) + b2
    a2 = sigmoid(z2)
    return a2

# PyTorch implementation

import torch
import torch.nn as nn

class SimpleNN(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super().__init__()
        self.fc1 = nn.Linear(input_dim, hidden_dim)
        self.act = nn.ReLU()
        self.fc2 = nn.Linear(hidden_dim, output_dim)

    def forward(self, x):
        return self.fc2(self.act(self.fc1(x)))

# Keras/TensorFlow implementation

import tensorflow as tf
from tensorflow import keras

model = keras.Sequential([
    keras.layers.Dense(64, activation='relu', input_shape=(input_dim,)),
    keras.layers.Dense(output_dim, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy')

# Overfitting mitigation via early stopping

early_stop = tf.keras.callbacks.EarlyStopping(
    monitor='val_loss', 
    patience=3, 
    restore_best_weights=True
)
model.fit(X_train, y_train, epochs=50, validation_split=0.2, callbacks=[early_stop])

Key Files and Repository Structure

The Neural Networks section organizes content across specific directories:

• [lessons/3-NeuralNetworks/README.md](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/README.md) – Overview and topic index (lines 9-48)
• [lessons/3-NeuralNetworks/03-Perceptron/README.md](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/03-Perceptron/README.md) – Single-layer network theory and implementation
• [lessons/3-NeuralNetworks/04-OwnFramework/README.md](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/04-OwnFramework/README.md) – Custom multi-layer framework construction
• lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb – PyTorch fundamentals and GPU utilization
• lessons/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb – TensorFlow/Keras high-level API tutorial

Summary

• The Neural Networks section (Lessons 03-05) spans the lessons/3-NeuralNetworks/ directory with three progressive sub-modules.
• Perceptron (Lesson 03) establishes the biological and mathematical neuron model using single-layer linear classification.
• Multi-layered networks (Lesson 04) require building a custom framework to understand backpropagation and hidden layer mechanics.
• Frameworks (Lesson 05) introduces production tools: PyTorch for research flexibility and Keras/TensorFlow for rapid deployment.
• Overfitting concepts are integrated throughout, with specific mitigation techniques demonstrated in the framework notebooks.
• All source references are documented in [README.md](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/README.md) lines 45-48 and foundational context in lines 9-34.

Frequently Asked Questions

What is the difference between the Perceptron and multi-layered networks in Lesson 03 versus Lesson 04?

The Perceptron (Lesson 03) is a single-layer binary classifier that can only separate data with a straight line (linearly separable problems), while multi-layered networks (Lesson 04) introduce hidden layers capable of learning non-linear decision boundaries through backpropagation. The curriculum explicitly builds the custom framework in Lesson 04 to demonstrate how stacking layers and adding non-linear activation functions solves the XOR problem that the Perceptron cannot.

Should beginners start with PyTorch or Keras/TensorFlow according to the Microsoft AI For Beginners curriculum?

The repository provides equal weight to both frameworks in Lesson 05, presenting PyTorch in IntroPyTorch.ipynb for users who prefer explicit control over tensor operations and debugging, and Keras/TensorFlow in IntroKerasTF.ipynb for those who favor rapid prototyping with high-level abstractions. Both notebooks cover identical architectures, allowing learners to compare the imperative versus declarative programming paradigms directly.

How does the curriculum teach overfitting detection and prevention?

Overfitting is addressed in the framework notebooks (specifically referenced at line 48) through practical implementation of validation monitoring, early stopping callbacks, and regularization techniques. The IntroKerasTF.ipynb notebook demonstrates EarlyStopping callbacks that monitor validation loss, while the PyTorch notebook implements manual validation loops to illustrate the concept of training versus validation accuracy divergence.

What prerequisites are required before starting the Neural Networks section (Lessons 03-05)?

According to the foundational concepts covered in lines 9-34 of the README, learners should understand basic machine learning terminology, linear algebra (vectors, matrices), and calculus (derivatives) before attempting these lessons. The curriculum assumes familiarity with Python and NumPy, as evidenced by the code implementations requiring matrix multiplication and gradient calculations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}