Mathematical Foundations Covered in Phase 1 of the AI Engineering Curriculum

Phase 1 of the rohitg00/ai-engineering-from-scratch curriculum delivers 22 self-contained lessons covering linear algebra, calculus, probability, and optimization, all implemented from scratch using a "stdlib-first" philosophy to build intuition for modern AI systems.

The mathematical foundations covered in Phase 1 provide the essential toolkit for understanding machine learning algorithms at a fundamental level. This repository treats each concept as a hands-on lesson with documentation, runnable code, and tests located in phases/01-math-foundations/. Unlike courses that rely on black-box libraries, Phase 1 builds every concept from first principles to develop deep debugging and extension capabilities.

The Complete Phase 1 Curriculum Structure

Phase 1 – Math Foundations organizes 22 lessons into a logical progression from geometric intuition to advanced stochastic modeling. Each lesson resides in phases/01-math-foundations/ and contains an explainer (docs/en.md), implementation code, unit tests, and quizzes.

The curriculum sequence follows this path:

1. Linear-Algebra Intuition – Geometric interpretation of vectors and transforms
2. Vectors & Matrices Operations – Broadcasting, reshaping, and multiplication
3. Matrix Transformations – Rotations, scaling, and shearing in space
4. Calculus for ML – Derivatives and gradients driving learning algorithms
5. Chain-Rule & Autodiff – Automatic differentiation for back-propagation
6. Probability & Distributions – Random variables, PDFs, and expectations
7. Bayes Theorem – Probabilistic reasoning in noisy environments
8. Optimization – Gradient descent and convergence guarantees
9. Information Theory – Entropy, KL-divergence, and mutual information
10. Dimensionality Reduction – Principal component analysis (PCA)
11. Singular-Value Decomposition – Low-rank matrix approximations
12. Tensor Operations – Higher-order array mathematics
13. Numerical Stability – Floating-point error management
14. Norms & Distances – L₁, L₂, and similarity metrics
15. Statistics for ML – Hypothesis testing and confidence intervals
16. Sampling Methods – Monte Carlo and importance sampling
17. Linear Systems – Solving Ax = b and matrix factorization
18. Convex Optimization – Convex sets, functions, and duality
19. Complex Numbers – Complex arithmetic and phasors
20. Fourier Transform – Frequency-domain representations
21. Graph Theory – Adjacency matrices and graph-based learning
22. Stochastic Processes – Markov chains and random walks

Core Mathematical Domains in Phase 1

The 22 lessons span four critical domains that underpin modern AI systems.

Linear Algebra and Matrix Computations

Lessons 1–3, 10–12, 14, and 17 establish the geometric intuition behind vector spaces. The curriculum implements matrix transformations, SVD, and tensor operations without relying on high-level abstractions. In phases/01-math-foundations/01-linear-algebra-intuition/code/main.py, learners implement rotation matrices and visualize linear transforms to understand how neural networks manipulate high-dimensional data.

Calculus and Optimization Theory

Lessons 4–5 and 8, 18 focus on gradients and convergence. The chain-rule lesson implements automatic differentiation from scratch to demystify back-propagation. The optimization lessons cover gradient descent mechanics and convexity, implemented in pure Python to reveal the underlying mathematics that drive weight updates in deep learning.

Probability, Statistics, and Information Theory

Lessons 6–7, 9, 15–16 cover probabilistic modeling and uncertainty quantification. Bayes Theorem implementations teach inference in noisy environments, while information theory lessons cover entropy and KL-divergence for model evaluation. Sampling methods include Monte Carlo estimation techniques essential for modern generative models.

Advanced Mathematical Structures

Lessons 13, 17, 19–22 address numerical stability, complex analysis, and graph theory. The Fourier Transform lesson connects frequency-domain analysis to convolutional networks, while graph theory lessons implement adjacency matrices for graph-based learning algorithms. Numerical stability lessons in phases/01-math-foundations/13-numerical-stability/ teach conditioning and overflow management critical for production AI systems.

Hands-On Implementation Philosophy

The curriculum adheres to a "stdlib-first" approach, requiring implementations using only Python's standard library or NumPy where explicitly allowed. This methodology appears in every lesson directory, which contains:

• docs/en.md – Theoretical explanations
• code/main.py – From-scratch implementations
• code/tests/test_main.py – Unit tests for validation

This structure ensures learners understand why algorithms behave as they do, rather than treating them as black boxes.

Code Examples from Phase 1

The repository provides minimal, self-contained snippets illustrating core concepts.

Rotation matrices and linear transforms:

# 1️⃣ Linear‑algebra intuition – visualising a 2‑D rotation matrix

import numpy as np, matplotlib.pyplot as plt

θ = np.radians(30)                     # 30° rotation

R = np.array([[np.cos(θ), -np.sin(θ)],
              [np.sin(θ),  np.cos(θ)]])
vec = np.array([1, 0])                 # unit vector along x‑axis

rotated = R @ vec

plt.quiver(0, 0, *vec,   color='r', scale=1, scale_units='xy')
plt.quiver(0, 0, *rotated, color='b', scale=1, scale_units='xy')
plt.xlim(-1.5, 1.5); plt.ylim(-1.5, 1.5); plt.gca().set_aspect('equal')
plt.title('Rotation matrix acting on a vector')
plt.show()

Automatic differentiation fundamentals:

# 2️⃣ Chain‑rule & automatic differentiation (pure Python)

def f(x): return x**3 + 2*x
def df_dx(x): return 3*x**2 + 2          # analytical derivative

x = 1.5
print(f(x), df_dx(x))                    # → (5.375, 9.75)

Monte Carlo sampling methods:

# 3️⃣ Sampling – simple Monte‑Carlo estimate of π

import random, math
N = 100_000
inside = sum(1 for _ in range(N) if (random.random()**2 + random.random()**2) <= 1)
pi_est = 4 * inside / N
print('π ≈', pi_est, 'error =', abs(pi_est - math.pi))

Repository Structure and Key Files

The mathematical foundations covered in Phase 1 are organized in phases/01-math-foundations/ with 22 subdirectories following the naming convention XX-topic-name/. Critical files include:

• site/data.js (lines 132–231) – Enumerates all Phase 1 lesson URLs and metadata, with the Optimization lesson entry appearing at line 195
• site/figures-math.js – Interactive visualizations supporting geometric intuition lessons
• phases/01-math-foundations/01-linear-algebra-intuition/docs/en.md – Example documentation file
• phases/01-math-foundations/01-linear-algebra-intuition/code/main.py – Implementation example
• phases/01-math-foundations/01-linear-algebra-intuition/code/tests/test_main.py – Unit test example

Each lesson directory mirrors this structure, providing a consistent learning interface across all 22 mathematical topics.

Summary

• Phase 1 contains 22 self-contained lessons covering linear algebra, calculus, probability, optimization, and advanced topics from SVD to Fourier transforms.
• Every lesson includes documentation, code implementation, and tests in dedicated subdirectories of phases/01-math-foundations/.
• The curriculum employs a "stdlib-first" philosophy, implementing algorithms from scratch to build deep mathematical intuition.
• Key files include site/data.js (lesson registry), individual docs/en.md files (explanations), and code/main.py implementations.
• Topics progress logically from geometric intuition (vectors, matrices) through optimization theory to stochastic processes and graph theory.

Frequently Asked Questions

What specific math topics are included in Phase 1?

Phase 1 covers 22 distinct lessons including Linear-Algebra Intuition, Matrix Transformations, Calculus for ML, Chain-Rule & Autodiff, Probability & Distributions, Bayes Theorem, Optimization, Information Theory, Dimensionality Reduction, SVD, Tensor Operations, Numerical Stability, Norms & Distances, Statistics for ML, Sampling Methods, Linear Systems, Convex Optimization, Complex Numbers, Fourier Transform, Graph Theory, and Stochastic Processes.

How are the Phase 1 lessons structured in the repository?

Each lesson follows a standardized layout within phases/01-math-foundations/ containing an explainer document at docs/en.md, implementation code in code/main.py, and validation tests in code/tests/test_main.py. The complete lesson list is indexed in site/data.js at lines 132–231.

Does Phase 1 require external libraries like PyTorch or TensorFlow?

No. The curriculum follows a "stdlib-first" philosophy, implementing concepts using only Python's standard library or NumPy where explicitly permitted. This approach ensures learners understand the underlying mathematics without abstraction layers, making it distinct from framework-centric courses.

Where can I find the interactive visualizations for mathematical concepts?

Interactive figures supporting Phase 1 lessons are located in site/figures-math.js. These visualizations complement the static code examples in lessons like Linear-Algebra Intuition and Matrix Transformations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}