# How Diffusion Models Generate Images Through the Denoising Process in AI Engineering From Scratch

> Explore how diffusion models generate images via denoising in AI Engineering From Scratch. Learn the step-by-step process of predicting and removing noise to create clear data from random noise.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-06-12

---

**Diffusion models generate images by learning to reverse a gradual noising process, using a neural network to iteratively predict and subtract noise at each timestep until coherent data emerges from pure Gaussian noise.**

The rohitg00/ai-engineering-from-scratch repository contains a complete educational implementation of Denoising Diffusion Probabilistic Models (DDPM) that demonstrates how diffusion models generate images through this iterative denoising process. Located in `phases/08-generative-ai/06-diffusion-ddpm-from-scratch/`, this codebase provides a from-scratch Python implementation using a 1-D mixture of two Gaussians to illustrate the mathematical foundations that power modern image generation systems like Stable Diffusion.

## The Forward Noising Process

The diffusion process begins with a **forward Markov chain** that gradually corrupts clean data into pure Gaussian noise. According to the documentation in [`phases/08-generative-ai/06-diffusion-ddpm-from-scratch/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/08-generative-ai/06-diffusion-ddpm-from-scratch/docs/en.md) (lines 20-27), the implementation uses a fixed variance schedule where each timestep adds a small amount of noise according to `q(x_t | x_{t-1})`.

The cumulative effect after `t` steps follows a closed-form Gaussian distribution:

```

q(x_t | x_0) = N( √α̅_t · x_0 , (1-α̅_t)·I )

```

Where `α̅_t = ∏_{s=1..t}(1-β_s)` represents the cumulative product of noise schedule parameters, with `β_t` linearly increasing from 1e-4 to 0.02 across 40 timesteps. This schedule is generated by the `make_schedule(T)` function, allowing the model to compute noisy samples directly without iterative corruption.

## Neural Network Architecture for Noise Prediction

The core of the denoising process is a neural network `ε_θ(x_t, t)` that predicts the noise component from a noisy sample. In [`phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py) (lines 47-54), the repository implements this as a three-layer MLP initialized via `init_net()`.

The network accepts:
- The noisy sample `x_t` (1-D in the toy example, extensible to image tensors)
- A **sinusoidal timestep embedding** `t` encoded in `t_dim` dimensions

The `forward()` function processes these inputs through hidden layers to output the predicted noise `ε̂`. This architecture learns to approximate the reverse conditional distribution, effectively acting as a denoiser that estimates what noise was added at each specific timestep.

## Training the Denoiser with MSE Loss

Training optimizes the network to minimize the difference between predicted and actual noise. As implemented in [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) (lines 24-25), the **simple loss** objective follows the variational lower bound:

```

L_simple = E_{x0, t, ε} [‖ε – ε_θ( √α̅_t·x0 + √(1-α̅_t)·ε , t )‖² ]

```

The `train()` function implements this by:
1. Sampling a random timestep `t`
2. Generating the noisy sample using the closed-form forward equation
3. Computing the **mean squared error (MSE)** between the true noise `ε` and the network's prediction `ε̂`
4. Updating the MLP weights via gradient descent

This single-line loss computation is sufficient to train the model for thousands of steps, typically converging within 4000 iterations with a learning rate of 0.01.

## Reverse Denoising: From Noise to Image

During inference, the model reverses the diffusion process through iterative denoising. Starting from pure noise `x_T ~ N(0, I)`, the `sample()` function in [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py) (lines 28-40) applies the learned reverse transition repeatedly from `t = T` down to `1`.

Each reverse step computes the posterior mean using the predicted noise:

```

x_{t-1} = ( x_t – β_t / √(1-α̅_t) · ε̂ ) / √α_t

```

For stochasticity (except at the final step), the implementation adds fresh Gaussian noise:

```

if t > 0: x_{t-1} += √β_t · z   (z ∼ N(0, I))

```

This loop progressively transforms random noise into structured data, demonstrating how diffusion models generate images by reversing the corruption process one timestep at a time.

## Complete Implementation Walkthrough

The repository provides runnable scripts to train and sample from the DDPM.

**Training the model on 1-D data:**

```python

# From phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py

rng = random.Random(13)                # Deterministic RNG

T, t_dim, hidden = 40, 8, 24           # 40 timesteps, 8-dim embedding, 24 hidden units

_, alphas, alpha_bars = make_schedule(T)

net = init_net(1, t_dim, hidden, rng)  # 1-D input, sinusoidal embedding

train(net, alpha_bars, T, steps=4000, lr=0.01, t_dim=t_dim, rng=rng)

```

**Generating new samples via reverse diffusion:**

```python
samples = [
    sample(net, alphas, alpha_bars, T, t_dim, rng)   # Reverse diffusion loop

    for _ in range(500)
]
print(histogram(samples))   # Visualizes the bimodal distribution

```

**Running the complete demo:**

```bash
python3 phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py

```

**Key files in the implementation:**

- [`docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/docs/en.md): Mathematical documentation covering the forward schedule, reverse process, and loss derivation
- [`code/main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/code/main.py): Complete implementation containing `init_net()`, `train()`, and `sample()` functions
- [`outputs/skill-diffusion-trainer.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/outputs/skill-diffusion-trainer.md): Skill packaging for the diffusion trainer
- `assets/ddpm.svg`: Visual diagram of the forward and reverse processes

## Summary

- Diffusion models generate data by reversing a **forward noising process** that corrupts clean signals into Gaussian noise over `T` timesteps.
- The **neural network** (`ε_θ`) learns to predict the specific noise added at each timestep using a sinusoidal embedding of the timestep index.
- Training minimizes the **MSE between predicted and actual noise**, a simplification of the variational lower bound that requires only one line of code.
- The **reverse sampling process** iteratively denoises random inputs using the learned predictor, gradually transforming `x_T ~ N(0, I)` into coherent samples.
- The rohitg00/ai-engineering-from-scratch repository demonstrates these principles with a **1-D mixture of two Gaussians**, though the same algorithm scales to high-dimensional image generation.

## Frequently Asked Questions

### What is the difference between the forward and reverse processes in diffusion models?

The forward process is a fixed Markov chain that gradually adds Gaussian noise to data according to a variance schedule, eventually producing pure noise. The reverse process is learned by the neural network, which predicts the noise at each step and uses this prediction to iteratively denoise a random sample back into coherent data. As implemented in the repository, the forward process uses the closed-form equation `q(x_t | x_0)` while the reverse process employs the learned `ε_θ` network in the `sample()` function.

### Why does the DDPM implementation use a sinusoidal timestep embedding?

The neural network needs to know which timestep `t` it is denoising to make accurate predictions, as the amount and scale of noise varies across the diffusion process. The sinusoidal embedding (with dimension `t_dim=8` in the code) encodes the timestep into a continuous vector that the MLP can process, allowing the single network to handle all timesteps conditionally rather than training separate models for each step.

### How does the noise schedule affect image generation quality?

The noise schedule controls how quickly data transitions from clean to noisy during training. The repository uses a linear schedule from 1e-4 to 0.02 over 40 timesteps, which determines the values of `α_t` and `β_t` used in both the forward corruption and reverse denoising steps. An appropriate schedule ensures stable training and high-quality generation; the 1-D demonstration in [`main.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/main.py) shows that this schedule successfully preserves a bimodal distribution after training.

### Can this 1-D implementation scale to actual image generation?

Yes, the mathematical framework is identical to that used in Stable Diffusion and other image models. While the repository uses a simple three-layer MLP for 1-D data to demonstrate the core concepts, the same `train()` and `sample()` logic applies to high-dimensional tensors. Image diffusion models typically replace the MLP with U-Net architectures and add attention mechanisms, but the fundamental process of predicting noise `ε` and applying the reverse diffusion formula remains unchanged.