# How AI Engineering from Scratch Compares to PyTorch and scikit‑learn

> Discover how AI Engineering from Scratch builds a complete AI stack from first principles, offering a unique mental model before diving into PyTorch and scikit-learn abstractions. Learn AI fundamentally.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: comparison
- Published: 2026-08-28

---

**AI Engineering from Scratch teaches the complete AI stack by building every component from first principles before introducing third‑party libraries, creating a direct mental model that maps hand‑crafted implementations to PyTorch and scikit‑learn abstractions.**

The rohitg00/ai-engineering-from-scratch repository follows a strict **“Build It / Use It”** pattern: you first implement low‑level primitives in pure Python, then map each to its industrial‑strength counterpart. This approach bridges the gap between mathematical theory and production code, revealing exactly what high‑level frameworks hide behind their APIs.

## The Build It / Use It Pedagogy

The curriculum is organized into *Build It* lessons that construct tensors, back‑propagation, optimizers, and data loaders from scratch, followed by *Use It* lessons that translate these primitives to **PyTorch** or **scikit‑learn** equivalents. This methodology appears in core files such as [`phases/03-deep-learning-core/10-mini-framework/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/10-mini-framework/docs/en.md) and [`phases/03-deep-learning-core/11-intro-to-pytorch/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/11-intro-to-pytorch/docs/en.md).

The pedagogical goal is understanding *how* the math works and why each abstraction exists, rather than simply calling high‑level APIs.

## From‑Scratch Implementation vs. Production Frameworks

### Hand‑Crafted Mini‑Framework

In [`phases/03-deep-learning-core/10-mini-framework/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/10-mini-framework/docs/en.md), the curriculum implements a **mini‑framework** in approximately 500 lines of pure Python. This library defines:

- **`Module`** base class with abstract `forward` and `backward` methods
- **`Linear`** layers with manual weight initialization and gradient computation
- **Activation functions** (`ReLU`, `Sigmoid`) and regularization (`Dropout`, `BatchNorm`)
- **Optimizers** (`SGD`, `Adam`) with explicit state tracking and parameter updates
- **`DataLoader`** for batch iteration and shuffling

Every tensor operation and gradient calculation is explicit. For example, the `Linear` layer manually computes forward passes and backward derivatives without autograd assistance.

### PyTorch Equivalents

The *Introduction to PyTorch* lesson in [`phases/03-deep-learning-core/11-intro-to-pytorch/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/11-intro-to-pytorch/docs/en.md) maps the mini‑framework classes directly to PyTorch primitives:

- `Module` → `torch.nn.Module` (same interface for `forward`, `parameters`, `train`/`eval` modes)
- `Linear` → `torch.nn.Linear` (automatic parameter registration, no manual gradient bookkeeping)
- `Dropout/BatchNorm` → Native layers with automatic train/eval mode handling
- `Adam` → `torch.optim.Adam` (adaptive moments without manual state tracking)
- `DataLoader` → `torch.utils.data.DataLoader` (multi‑process loading and GPU acceleration)

The implementation shifts from manual backward methods to **tape‑based automatic differentiation**, eliminating explicit gradient bookkeeping.

### scikit‑learn Abstractions

For classical machine learning, lessons like [`phases/02-ml-fundamentals/02-linear-regression/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/02-ml-fundamentals/02-linear-regression/docs/en.md) demonstrate how the same problem—such as linear regression—can be solved three ways: NumPy implementation, scikit‑learn one‑liner (`LinearRegression().fit(X, y)`), and discussion of edge cases the library handles automatically (numerical stability, regularization).

## Side‑by‑Side Technical Comparison

| Aspect | AI Engineering from Scratch | PyTorch | scikit‑learn |
|--------|----------------------------|---------|--------------|
| **Goal** | Understand mathematical foundations and infrastructure | Production‑ready, GPU‑accelerated deep learning | High‑level API for classical ML pipelines |
| **Implementation** | ~500 LOC hand‑written mini‑framework (`Module`, `Linear`, `Sequential`) | `torch.nn.Module`, `torch.nn.Sequential`, autograd | Ready‑made estimators (`LinearRegression`, `LogisticRegression`) |
| **Autograd** | Manual `backward` methods per layer | Automatic differentiation via `loss.backward()` | No explicit gradient handling; algorithms pre‑implemented |
| **Device Support** | CPU‑only pure Python | Transparent CPU/GPU via `torch.device` and `.to()` | CPU‑only; delegates to NumPy/SciPy |
| **Performance** | ~300 seconds/epoch (MNIST MLP on CPU) | ~0.5 seconds/epoch (CPU), ~5 seconds/epoch (GPU) | Competitive for small‑to‑medium datasets via vectorized NumPy |
| **Educational Value** | Explicit forward/right‑hand side of every equation; visible weight initialization and gradient flow | Immediate comparison reveals hidden mechanics (autograd, device placement) | Understanding when to replace custom code with battle‑tested implementations |

## Practical Code Comparison: Circle Classification

The following examples solve identical binary classification problems (circle data) at three abstraction levels, demonstrating the mental mapping taught in the curriculum.

### 1. Hand‑Crafted Framework

```python
import random, math

# Module, Linear, ReLU, Sigmoid, BCELoss, Adam, DataLoader defined in lesson

def make_circle_data(n=500):
    random.seed(42)
    data = []
    for _ in range(n):
        x = random.uniform(-2, 2)
        y = random.uniform(-2, 2)
        label = 1.0 if x*x + y*y < 1.5 else 0.0
        data.append(([x, y], [label]))
    return data

model = Sequential(
    Linear(2, 16), ReLU(),
    Linear(16, 8), ReLU(),
    Linear(8, 1), Sigmoid(),
)
criterion = BCELoss()
optimizer = Adam(model.parameters(), lr=0.01)
loader = DataLoader(make_circle_data(), batch_size=16, shuffle=True)

model.train()
for epoch in range(100):
    for inputs, targets in loader:
        optimizer.zero_grad()
        loss = 0
        grads = []
        for x, t in zip(inputs, targets):
            pred = model.forward(x)
            loss += criterion(pred, t)
            grads.append(criterion.backward())
        for grad in grads:
            model.backward(grad)
        optimizer.step()

```

### 2. PyTorch Implementation

```python
import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

def make_circle_data_tensor():
    torch.manual_seed(42)
    X = torch.empty(500, 2).uniform_(-2, 2)
    y = (X[:,0]**2 + X[:,1]**2 < 1.5).float().unsqueeze(1)
    return TensorDataset(X, y)

model = nn.Sequential(
    nn.Linear(2, 16), nn.ReLU(),
    nn.Linear(16, 8), nn.ReLU(),
    nn.Linear(8, 1), nn.Sigmoid(),
)
criterion = nn.BCELoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)
loader = DataLoader(make_circle_data_tensor(), batch_size=16, shuffle=True)

model.train()
for epoch in range(100):
    for x, t in loader:
        optimizer.zero_grad()
        pred = model(x)
        loss = criterion(pred, t)
        loss.backward()
        optimizer.step()

```

### 3. scikit‑learn Approach

```python
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

def make_circle_data_np():
    np.random.seed(42)
    X = np.random.uniform(-2, 2, size=(500, 2))
    y = (X[:,0]**2 + X[:,1]**2 < 1.5).astype(int)
    return X, y

X, y = make_circle_data_np()
clf = LogisticRegression(max_iter=1000)
clf.fit(X, y)

pred = clf.predict(X)
print('Training accuracy:', accuracy_score(y, pred))

```

The progression illustrates how **AI Engineering from Scratch** makes every forward and backward step explicit, while PyTorch hides gradients behind `autograd` and scikit‑learn abstracts the entire pipeline into a single estimator call.

## When to Use Each Approach

**Use the mini‑framework** when you need to understand gradient flow, debug vanishing gradients, or implement custom layers not available in standard libraries. The explicit code in [`phases/03-deep-learning-core/10-mini-framework/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/10-mini-framework/docs/en.md) reveals exactly why `optimizer.zero_grad()` matters and how weight initialization affects convergence.

**Use PyTorch** when you need GPU acceleration, automatic differentiation, or production deployment. As shown in [`phases/03-deep-learning-core/11-intro-to-pytorch/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/11-intro-to-pytorch/docs/en.md), PyTorch provides optimized C++/CUDA kernels that run orders of magnitude faster than pure Python implementations.

**Use scikit‑learn** for classical machine learning tasks on small‑to‑medium datasets where you need robust, battle‑tested implementations of algorithms like linear regression, logistic regression, or TF‑IDF vectorization (covered in [`phases/05-nlp-foundations-to-advanced/02-bag-of-words-tfidf/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/05-nlp-foundations-to-advanced/02-bag-of-words-tfidf/docs/en.md)).

## Summary

- **AI Engineering from Scratch** builds a mental model by implementing tensors, back‑propagation, and optimizers in pure Python before introducing external libraries.
- The **Build It / Use It** pattern creates one‑to‑one mappings between custom `Module` classes and `torch.nn.Module`, or between NumPy implementations and scikit‑learn estimators.
- **Performance trade‑offs** are explicit: the educational framework runs ~300 seconds per MNIST epoch versus PyTorch’s ~0.5 seconds on CPU, illustrating the cost of abstraction.
- **Autograd mechanics** are demystified by writing manual `backward` methods first, making `loss.backward()` in PyTorch transparent rather than magical.
- **File references** such as [`phases/03-deep-learning-core/10-mini-framework/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/10-mini-framework/docs/en.md) provide the complete source code for the hand‑crafted framework and its PyTorch equivalents.

## Frequently Asked Questions

### Does AI Engineering from Scratch replace PyTorch and scikit‑learn?

No. The curriculum teaches you the foundations so you can use PyTorch and scikit‑learn effectively. By implementing `Linear`, `BatchNorm`, and `Adam` from scratch in [`phases/03-deep-learning-core/10-mini-framework/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/03-deep-learning-core/10-mini-framework/docs/en.md), you gain the intuition to debug framework code and understand what happens inside `torch.optim.Adam` or `sklearn.linear_model.LinearRegression`.

### How does the mini‑framework handle automatic differentiation?

It does not. You write explicit `backward` methods for every layer that compute gradients manually using the chain rule. This contrasts with PyTorch’s tape‑based autograd, which records operations during the forward pass and replays them during `loss.backward()`. The manual approach appears in the hand‑crafted `Module` base class defined in the deep learning core lessons.

### Can I use the mini‑framework for production workloads?

No. The pure Python implementation lacks GPU support and optimized kernels, resulting in performance approximately 600× slower than PyTorch on CPU for MNIST classification. The framework serves educational purposes only, explicitly designed to answer “what does the library hide?” before you transition to industrial frameworks.

### What is the “Build It / Use It” pattern?

This pedagogical structure requires you to construct a component—such as a `DataLoader` or TF‑IDF vectorizer—from first principles, then immediately compare it to the PyTorch or scikit‑learn equivalent. Lessons like [`phases/05-nlp-foundations-to-advanced/02-bag-of-words-tfidf/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/05-nlp-foundations-to-advanced/02-bag-of-words-tfidf/docs/en.md) show the from‑scratch implementation followed by the three‑line scikit‑learn solution, highlighting numerical stability and edge cases the library handles automatically.