# Mathematical Foundations Covered in Phase 1 of the AI Engineering Curriculum

> Discover the mathematical foundations linear algebra calculus probability and optimization in Phase 1 of the AI Engineering curriculum. Learn from scratch with practical implementations.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-07-27

---

**Phase 1 of the rohitg00/ai-engineering-from-scratch curriculum delivers 22 self-contained lessons covering linear algebra, calculus, probability, and optimization, all implemented from scratch using a "stdlib-first" philosophy to build intuition for modern AI systems.**

The mathematical foundations covered in Phase 1 provide the essential toolkit for understanding machine learning algorithms at a fundamental level. This repository treats each concept as a hands-on lesson with documentation, runnable code, and tests located in `phases/01-math-foundations/`. Unlike courses that rely on black-box libraries, Phase 1 builds every concept from first principles to develop deep debugging and extension capabilities.

## The Complete Phase 1 Curriculum Structure

Phase 1 – *Math Foundations* organizes 22 lessons into a logical progression from geometric intuition to advanced stochastic modeling. Each lesson resides in `phases/01-math-foundations/` and contains an explainer ([`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md)), implementation code, unit tests, and quizzes.

The curriculum sequence follows this path:

1. **Linear-Algebra Intuition** – Geometric interpretation of vectors and transforms
2. **Vectors & Matrices Operations** – Broadcasting, reshaping, and multiplication
3. **Matrix Transformations** – Rotations, scaling, and shearing in space
4. **Calculus for ML** – Derivatives and gradients driving learning algorithms
5. **Chain-Rule & Autodiff** – Automatic differentiation for back-propagation
6. **Probability & Distributions** – Random variables, PDFs, and expectations
7. **Bayes Theorem** – Probabilistic reasoning in noisy environments
8. **Optimization** – Gradient descent and convergence guarantees
9. **Information Theory** – Entropy, KL-divergence, and mutual information
10. **Dimensionality Reduction** – Principal component analysis (PCA)
11. **Singular-Value Decomposition** – Low-rank matrix approximations
12. **Tensor Operations** – Higher-order array mathematics
13. **Numerical Stability** – Floating-point error management
14. **Norms & Distances** – L₁, L₂, and similarity metrics
15. **Statistics for ML** – Hypothesis testing and confidence intervals
16. **Sampling Methods** – Monte Carlo and importance sampling
17. **Linear Systems** – Solving Ax = b and matrix factorization
18. **Convex Optimization** – Convex sets, functions, and duality
19. **Complex Numbers** – Complex arithmetic and phasors
20. **Fourier Transform** – Frequency-domain representations
21. **Graph Theory** – Adjacency matrices and graph-based learning
22. **Stochastic Processes** – Markov chains and random walks

## Core Mathematical Domains in Phase 1

The 22 lessons span four critical domains that underpin modern AI systems.

### Linear Algebra and Matrix Computations

Lessons 1–3, 10–12, 14, and 17 establish the **geometric intuition** behind vector spaces. The curriculum implements matrix transformations, SVD, and tensor operations without relying on high-level abstractions. In [`phases/01-math-foundations/01-linear-algebra-intuition/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/01-math-foundations/01-linear-algebra-intuition/code/main.py), learners implement rotation matrices and visualize linear transforms to understand how neural networks manipulate high-dimensional data.

### Calculus and Optimization Theory

Lessons 4–5 and 8, 18 focus on **gradients and convergence**. The chain-rule lesson implements automatic differentiation from scratch to demystify back-propagation. The optimization lessons cover gradient descent mechanics and convexity, implemented in pure Python to reveal the underlying mathematics that drive weight updates in deep learning.

### Probability, Statistics, and Information Theory

Lessons 6–7, 9, 15–16 cover **probabilistic modeling** and uncertainty quantification. Bayes Theorem implementations teach inference in noisy environments, while information theory lessons cover entropy and KL-divergence for model evaluation. Sampling methods include Monte Carlo estimation techniques essential for modern generative models.

### Advanced Mathematical Structures

Lessons 13, 17, 19–22 address **numerical stability**, complex analysis, and graph theory. The Fourier Transform lesson connects frequency-domain analysis to convolutional networks, while graph theory lessons implement adjacency matrices for graph-based learning algorithms. Numerical stability lessons in `phases/01-math-foundations/13-numerical-stability/` teach conditioning and overflow management critical for production AI systems.

## Hands-On Implementation Philosophy

The curriculum adheres to a **"stdlib-first"** approach, requiring implementations using only Python's standard library or NumPy where explicitly allowed. This methodology appears in every lesson directory, which contains:

- [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) – Theoretical explanations
- [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) – From-scratch implementations
- [`code/tests/test_main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/tests/test_main.py) – Unit tests for validation

This structure ensures learners understand why algorithms behave as they do, rather than treating them as black boxes.

## Code Examples from Phase 1

The repository provides minimal, self-contained snippets illustrating core concepts.

Rotation matrices and linear transforms:

```python

# 1️⃣ Linear‑algebra intuition – visualising a 2‑D rotation matrix

import numpy as np, matplotlib.pyplot as plt

θ = np.radians(30)                     # 30° rotation

R = np.array([[np.cos(θ), -np.sin(θ)],
              [np.sin(θ),  np.cos(θ)]])
vec = np.array([1, 0])                 # unit vector along x‑axis

rotated = R @ vec

plt.quiver(0, 0, *vec,   color='r', scale=1, scale_units='xy')
plt.quiver(0, 0, *rotated, color='b', scale=1, scale_units='xy')
plt.xlim(-1.5, 1.5); plt.ylim(-1.5, 1.5); plt.gca().set_aspect('equal')
plt.title('Rotation matrix acting on a vector')
plt.show()

```

Automatic differentiation fundamentals:

```python

# 2️⃣ Chain‑rule & automatic differentiation (pure Python)

def f(x): return x**3 + 2*x
def df_dx(x): return 3*x**2 + 2          # analytical derivative

x = 1.5
print(f(x), df_dx(x))                    # → (5.375, 9.75)

```

Monte Carlo sampling methods:

```python

# 3️⃣ Sampling – simple Monte‑Carlo estimate of π

import random, math
N = 100_000
inside = sum(1 for _ in range(N) if (random.random()**2 + random.random()**2) <= 1)
pi_est = 4 * inside / N
print('π ≈', pi_est, 'error =', abs(pi_est - math.pi))

```

## Repository Structure and Key Files

The mathematical foundations covered in Phase 1 are organized in `phases/01-math-foundations/` with 22 subdirectories following the naming convention `XX-topic-name/`. Critical files include:

- **[`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js)** (lines 132–231) – Enumerates all Phase 1 lesson URLs and metadata, with the Optimization lesson entry appearing at line 195
- **[`site/figures-math.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/figures-math.js)** – Interactive visualizations supporting geometric intuition lessons
- **[`phases/01-math-foundations/01-linear-algebra-intuition/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/01-math-foundations/01-linear-algebra-intuition/docs/en.md)** – Example documentation file
- **[`phases/01-math-foundations/01-linear-algebra-intuition/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/01-math-foundations/01-linear-algebra-intuition/code/main.py)** – Implementation example
- **[`phases/01-math-foundations/01-linear-algebra-intuition/code/tests/test_main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/01-math-foundations/01-linear-algebra-intuition/code/tests/test_main.py)** – Unit test example

Each lesson directory mirrors this structure, providing a consistent learning interface across all 22 mathematical topics.

## Summary

- Phase 1 contains **22 self-contained lessons** covering linear algebra, calculus, probability, optimization, and advanced topics from SVD to Fourier transforms.
- Every lesson includes **documentation, code implementation, and tests** in dedicated subdirectories of `phases/01-math-foundations/`.
- The curriculum employs a **"stdlib-first" philosophy**, implementing algorithms from scratch to build deep mathematical intuition.
- Key files include [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) (lesson registry), individual [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md) files (explanations), and [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) implementations.
- Topics progress logically from **geometric intuition** (vectors, matrices) through **optimization theory** to **stochastic processes** and **graph theory**.

## Frequently Asked Questions

### What specific math topics are included in Phase 1?

Phase 1 covers 22 distinct lessons including Linear-Algebra Intuition, Matrix Transformations, Calculus for ML, Chain-Rule & Autodiff, Probability & Distributions, Bayes Theorem, Optimization, Information Theory, Dimensionality Reduction, SVD, Tensor Operations, Numerical Stability, Norms & Distances, Statistics for ML, Sampling Methods, Linear Systems, Convex Optimization, Complex Numbers, Fourier Transform, Graph Theory, and Stochastic Processes.

### How are the Phase 1 lessons structured in the repository?

Each lesson follows a standardized layout within `phases/01-math-foundations/` containing an explainer document at [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md), implementation code in [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py), and validation tests in [`code/tests/test_main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/tests/test_main.py). The complete lesson list is indexed in [`site/data.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/data.js) at lines 132–231.

### Does Phase 1 require external libraries like PyTorch or TensorFlow?

No. The curriculum follows a **"stdlib-first"** philosophy, implementing concepts using only Python's standard library or NumPy where explicitly permitted. This approach ensures learners understand the underlying mathematics without abstraction layers, making it distinct from framework-centric courses.

### Where can I find the interactive visualizations for mathematical concepts?

Interactive figures supporting Phase 1 lessons are located in [`site/figures-math.js`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/site/figures-math.js). These visualizations complement the static code examples in lessons like Linear-Algebra Intuition and Matrix Transformations.