How Diffusion Models Generate Images Through the Denoising Process in AI Engineering From Scratch

Diffusion models generate images by learning to reverse a gradual noising process, using a neural network to iteratively predict and subtract noise at each timestep until coherent data emerges from pure Gaussian noise.

The rohitg00/ai-engineering-from-scratch repository contains a complete educational implementation of Denoising Diffusion Probabilistic Models (DDPM) that demonstrates how diffusion models generate images through this iterative denoising process. Located in phases/08-generative-ai/06-diffusion-ddpm-from-scratch/, this codebase provides a from-scratch Python implementation using a 1-D mixture of two Gaussians to illustrate the mathematical foundations that power modern image generation systems like Stable Diffusion.

The Forward Noising Process

The diffusion process begins with a forward Markov chain that gradually corrupts clean data into pure Gaussian noise. According to the documentation in phases/08-generative-ai/06-diffusion-ddpm-from-scratch/docs/en.md (lines 20-27), the implementation uses a fixed variance schedule where each timestep adds a small amount of noise according to q(x_t | x_{t-1}).

The cumulative effect after t steps follows a closed-form Gaussian distribution:

q(x_t | x_0) = N( √α̅_t · x_0 , (1-α̅_t)·I )

Where α̅_t = ∏_{s=1..t}(1-β_s) represents the cumulative product of noise schedule parameters, with β_t linearly increasing from 1e-4 to 0.02 across 40 timesteps. This schedule is generated by the make_schedule(T) function, allowing the model to compute noisy samples directly without iterative corruption.

Neural Network Architecture for Noise Prediction

The core of the denoising process is a neural network ε_θ(x_t, t) that predicts the noise component from a noisy sample. In phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py (lines 47-54), the repository implements this as a three-layer MLP initialized via init_net().

The network accepts:

• The noisy sample x_t (1-D in the toy example, extensible to image tensors)
• A sinusoidal timestep embedding t encoded in t_dim dimensions

The forward() function processes these inputs through hidden layers to output the predicted noise ε̂. This architecture learns to approximate the reverse conditional distribution, effectively acting as a denoiser that estimates what noise was added at each specific timestep.

Training the Denoiser with MSE Loss

Training optimizes the network to minimize the difference between predicted and actual noise. As implemented in code/main.py (lines 24-25), the simple loss objective follows the variational lower bound:

L_simple = E_{x0, t, ε} [‖ε – ε_θ( √α̅_t·x0 + √(1-α̅_t)·ε , t )‖² ]

The train() function implements this by:

1. Sampling a random timestep t
2. Generating the noisy sample using the closed-form forward equation
3. Computing the mean squared error (MSE) between the true noise ε and the network's prediction ε̂
4. Updating the MLP weights via gradient descent

This single-line loss computation is sufficient to train the model for thousands of steps, typically converging within 4000 iterations with a learning rate of 0.01.

Reverse Denoising: From Noise to Image

During inference, the model reverses the diffusion process through iterative denoising. Starting from pure noise x_T ~ N(0, I), the sample() function in code/main.py (lines 28-40) applies the learned reverse transition repeatedly from t = T down to 1.

Each reverse step computes the posterior mean using the predicted noise:

x_{t-1} = ( x_t – β_t / √(1-α̅_t) · ε̂ ) / √α_t

For stochasticity (except at the final step), the implementation adds fresh Gaussian noise:

if t > 0: x_{t-1} += √β_t · z   (z ∼ N(0, I))

This loop progressively transforms random noise into structured data, demonstrating how diffusion models generate images by reversing the corruption process one timestep at a time.

Complete Implementation Walkthrough

The repository provides runnable scripts to train and sample from the DDPM.

Training the model on 1-D data:

# From phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py

rng = random.Random(13)                # Deterministic RNG

T, t_dim, hidden = 40, 8, 24           # 40 timesteps, 8-dim embedding, 24 hidden units

_, alphas, alpha_bars = make_schedule(T)

net = init_net(1, t_dim, hidden, rng)  # 1-D input, sinusoidal embedding

train(net, alpha_bars, T, steps=4000, lr=0.01, t_dim=t_dim, rng=rng)

Generating new samples via reverse diffusion:

samples = [
    sample(net, alphas, alpha_bars, T, t_dim, rng)   # Reverse diffusion loop

    for _ in range(500)
]
print(histogram(samples))   # Visualizes the bimodal distribution

Running the complete demo:

python3 phases/08-generative-ai/06-diffusion-ddpm-from-scratch/code/main.py

Key files in the implementation:

• docs/en.md: Mathematical documentation covering the forward schedule, reverse process, and loss derivation
• code/main.py: Complete implementation containing init_net(), train(), and sample() functions
• outputs/skill-diffusion-trainer.md: Skill packaging for the diffusion trainer
• assets/ddpm.svg: Visual diagram of the forward and reverse processes

Summary

• Diffusion models generate data by reversing a forward noising process that corrupts clean signals into Gaussian noise over T timesteps.
• The neural network (ε_θ) learns to predict the specific noise added at each timestep using a sinusoidal embedding of the timestep index.
• Training minimizes the MSE between predicted and actual noise, a simplification of the variational lower bound that requires only one line of code.
• The reverse sampling process iteratively denoises random inputs using the learned predictor, gradually transforming x_T ~ N(0, I) into coherent samples.
• The rohitg00/ai-engineering-from-scratch repository demonstrates these principles with a 1-D mixture of two Gaussians, though the same algorithm scales to high-dimensional image generation.

Frequently Asked Questions

What is the difference between the forward and reverse processes in diffusion models?

The forward process is a fixed Markov chain that gradually adds Gaussian noise to data according to a variance schedule, eventually producing pure noise. The reverse process is learned by the neural network, which predicts the noise at each step and uses this prediction to iteratively denoise a random sample back into coherent data. As implemented in the repository, the forward process uses the closed-form equation q(x_t | x_0) while the reverse process employs the learned ε_θ network in the sample() function.

Why does the DDPM implementation use a sinusoidal timestep embedding?

The neural network needs to know which timestep t it is denoising to make accurate predictions, as the amount and scale of noise varies across the diffusion process. The sinusoidal embedding (with dimension t_dim=8 in the code) encodes the timestep into a continuous vector that the MLP can process, allowing the single network to handle all timesteps conditionally rather than training separate models for each step.

How does the noise schedule affect image generation quality?

The noise schedule controls how quickly data transitions from clean to noisy during training. The repository uses a linear schedule from 1e-4 to 0.02 over 40 timesteps, which determines the values of α_t and β_t used in both the forward corruption and reverse denoising steps. An appropriate schedule ensures stable training and high-quality generation; the 1-D demonstration in main.py shows that this schedule successfully preserves a bimodal distribution after training.

Can this 1-D implementation scale to actual image generation?

Yes, the mathematical framework is identical to that used in Stable Diffusion and other image models. While the repository uses a simple three-layer MLP for 1-D data to demonstrate the core concepts, the same train() and sample() logic applies to high-dimensional tensors. Image diffusion models typically replace the MLP with U-Net architectures and add attention mechanisms, but the fundamental process of predicting noise ε and applying the reverse diffusion formula remains unchanged.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}