How to Build Generative Adversarial Networks (GANs) for Image Generation Using PyTorch

Build a DCGAN in PyTorch by defining a Generator network that upsamples noise vectors into images and a Discriminator network that classifies real versus fake, then train both with alternating Adam optimizers using binary cross-entropy loss.

A Generative Adversarial Network (GAN) synthesizes realistic images by pitting two neural networks against each other in a zero-sum game. The AI-For-Beginners repository by Microsoft provides a complete, beginner-friendly implementation that demonstrates this architecture on the MNIST dataset. This guide walks through the exact code structure, training dynamics, and practical adaptations you need to build GANs for image generation using PyTorch.

---

GAN Architecture Components

Every GAN consists of two competing networks. According to the source code in lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb, these components are implemented as separate nn.Module classes.

Generator Network

The Generator maps a random latent vector z into a synthetic image. In the repository's implementation (around line 103), this uses transposed convolutions to progressively upsample the tensor:

class Generator(nn.Module):
    def __init__(self, nz=100, ngf=64, nc=1):
        super(Generator, self).__init__()
        self.main = nn.Sequential(
            # input is Z, go through a transposed conv

            nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False),
            nn.BatchNorm2d(ngf * 8),
            nn.ReLU(True),

            nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),

            nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),

            nn.ConvTranspose2d(ngf * 2, nc, 4, 2, 1, bias=False),
            nn.Tanh()  # output in [-1, 1]

        )

Key architectural decisions in this generator:

• Input: 100-dimensional noise vector (nz = 100), reshaped to (batch, 100, 1, 1)
• Upsampling path: Four ConvTranspose2d layers expand 1×1 → 4×4 → 7×7 → 14×14 → 28×28
• BatchNorm2d: Stabilizes training by normalizing intermediate feature maps
• Tanh output: Constrains generated pixels to [-1, 1], matching normalized image data

Discriminator Network

The Discriminator classifies images as real or fake. The implementation (around line 163) uses strided convolutions to downsample:

class Discriminator(nn.Module):
    def __init__(self, ndf=64, nc=1):
        super(Discriminator, self).__init__()
        self.main = nn.Sequential(
            # input is (nc) x 28 x 28

            nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),

            nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 2),
            nn.LeakyReLU(0.2, inplace=True),

            nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 4),
            nn.LeakyReLU(0.2, inplace=True),

            nn.Conv2d(ndf * 4, 1, 4, 1, 0, bias=False),
            nn.Sigmoid()   # probability of being real

        )

Critical discriminator design choices:

• LeakyReLU(0.2): Allows small negative gradients, preventing dead neurons
• No BatchNorm on first layer: Preserves raw input statistics for more stable training
• Sigmoid output: Produces probability ∈ [0, 1] for binary classification

---

Loss Functions and Training Objectives

The GAN training objective is a binary cross-entropy (BCE) minimax game. Both networks share the same loss function but optimize opposing goals:

criterion = nn.BCELoss()

Network	Objective	Mathematical Form
Discriminator	Maximize log(D(x)) + log(1 - D(G(z)))	Correctly classify real and fake
Generator	Maximize log(D(G(z)))	Fool discriminator into labeling fakes as real

The generator's loss uses real labels for its fake outputs—this is the "adversarial" trick that trains G to produce convincing forgeries.

---

Implementing the GAN Training Loop in PyTorch

The training loop (lines 200-300 in GANPyTorch.ipynb) alternates between updating the discriminator and generator. This separation prevents one network from dominating too early:

for epoch in range(epochs):
    for real_imgs, _ in dataloader:
        # Move to GPU if available

        real_imgs = real_imgs.to(device)
        batch_size = real_imgs.size(0)
        
        # Create labels

        real_labels = torch.full((batch_size, 1), 1., device=device)
        fake_labels = torch.full((batch_size, 1), 0., device=device)

        # -------------------------------------------------

        # Train Discriminator: maximize log(D(x)) + log(1-D(G(z)))

        # -------------------------------------------------

        
        # Forward pass with real images

        output_real = netD(real_imgs)
        loss_real = criterion(output_real, real_labels)
        
        # Generate fake images (detached from generator graph)

        noise = torch.randn(batch_size, nz, 1, 1, device=device)
        fake_imgs = netG(noise)
        
        # Forward pass with fake images

        output_fake = netD(fake_imgs.detach())
        loss_fake = criterion(output_fake, fake_labels)
        
        # Combined discriminator loss and update

        d_loss = loss_real + loss_fake
        optimizerD.zero_grad()
        d_loss.backward()
        optimizerD.step()

        # -------------------------------------------------

        # Train Generator: maximize log(D(G(z)))

        # -------------------------------------------------

        
        # Generator wants discriminator to label fakes as real

        output = netD(fake_imgs)
        g_loss = criterion(output, real_labels)  # Note: real_labels!

        
        optimizerG.zero_grad()
        g_loss.backward()
        optimizerG.step()

Critical training mechanics:

• .detach(): Prevents gradients from flowing into the generator during discriminator updates
• Separate optimizers: optimizerD and optimizerG each manage one network's parameters
• Adam with β₁=0.5: The repository uses betas=(0.5, 0.999) to handle unstable GAN gradients

---

Complete Standalone GAN Implementation

Below is a minimal, runnable script adapted from GANPyTorch.ipynb that you can execute on any image dataset. The architecture matches the repository's DCGAN implementation:

# -------------------------------------------------

#  Minimal DCGAN for image generation - PyTorch

# -------------------------------------------------

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# ------------------- Hyperparameters -------------------

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
nz = 100              # latent dimension

ngf = 64              # generator feature maps

ndf = 64              # discriminator feature maps

nc = 1                # image channels (1 for MNIST, 3 for RGB)

batch_size = 128
lr = 2e-4
epochs = 50

# ------------------- Data Loading ----------------------

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize([0.5], [0.5])   # scales to [-1, 1]

])

train_loader = DataLoader(
    datasets.MNIST(root='./data', train=True, download=True, transform=transform),
    batch_size=batch_size, shuffle=True, num_workers=2
)

# ------------------- Generator -------------------------

class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.main = nn.Sequential(
            # input: (nz, 1, 1)

            nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False),
            nn.BatchNorm2d(ngf * 8),
            nn.ReLU(True),
            # state: (ngf*8, 4, 4)

            nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
            # state: (ngf*4, 7, 7)

            nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            # state: (ngf*2, 14, 14)

            nn.ConvTranspose2d(ngf * 2, nc, 4, 2, 1, bias=False),
            nn.Tanh()
            # output: (nc, 28, 28)

        )
    
    def forward(self, z):
        return self.main(z)

# ------------------- Discriminator ---------------------

class Discriminator(nn.Module):
    def __init__(self):
        super().__init__()
        self.main = nn.Sequential(
            # input: (nc, 28, 28)

            nn.Conv2d(nc, ndf, 4, 2, 1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),
            # state: (ndf, 14, 14)

            nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 2),
            nn.LeakyReLU(0.2, inplace=True),
            # state: (ndf*2, 7, 7)

            nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ndf * 4),
            nn.LeakyReLU(0.2, inplace=True),
            # state: (ndf*4, 3, 3)

            nn.Conv2d(ndf * 4, 1, 4, 1, 0, bias=False),
            nn.Sigmoid()
            # output: (1, 1, 1) -> scalar probability

        )
    
    def forward(self, x):
        return self.main(x)

# ------------------- Initialization --------------------

netG = Generator().to(device)
netD = Discriminator().to(device)

# Apply custom weight initialization (as in DCGAN paper)

def weights_init(m):
    classname = m.__class__.__name__
    if classname.find('Conv') != -1:
        nn.init.normal_(m.weight.data, 0.0, 0.02)
    elif classname.find('BatchNorm') != -1:
        nn.init.normal_(m.weight.data, 1.0, 0.02)
        nn.init.constant_(m.bias.data, 0)

netG.apply(weights_init)
netD.apply(weights_init)

# Loss and optimizers

criterion = nn.BCELoss()
optimizerD = optim.Adam(netD.parameters(), lr=lr, betas=(0.5, 0.999))
optimizerG = optim.Adam(netG.parameters(), lr=lr, betas=(0.5, 0.999))

# ------------------- Training Loop ---------------------

for epoch in range(epochs):
    for i, (real_imgs, _) in enumerate(train_loader):
        real_imgs = real_imgs.to(device)
        cur_batch_size = real_imgs.size(0)
        
        real_labels = torch.full((cur_batch_size, 1, 1, 1), 1., device=device)
        fake_labels = torch.full((cur_batch_size, 1, 1, 1), 0., device=device)
        
        # ------- Update Discriminator -------

        # Real batch

        netD.zero_grad()
        output_real = netD(real_imgs)
        loss_real = criterion(output_real, real_labels)
        
        # Fake batch

        noise = torch.randn(cur_batch_size, nz, 1, 1, device=device)
        fake_imgs = netG(noise)
        output_fake = netD(fake_imgs.detach())
        loss_fake = criterion(output_fake, fake_labels)
        
        # Combined and backward

        loss_D = loss_real + loss_fake
        loss_D.backward()
        optimizerD.step()
        
        # ------- Update Generator -------

        netG.zero_grad()
        output = netD(fake_imgs)
        loss_G = criterion(output, real_labels)
        loss_G.backward()
        optimizerG.step()
        
        # Print progress

        if i % 100 == 0:
            print(f'Epoch [{epoch}/{epochs}] Batch [{i}/{len(train_loader)}] '
                  f'Loss_D: {loss_D.item():.4f} Loss_G: {loss_G.item():.4f}')

print("Training complete!")

---

Adapting to Different Image Datasets

To build GANs for image generation beyond MNIST, modify these parameters:

Parameter	MNIST	CIFAR-10	CelebA (64×64)
nc (channels)	1	3	3
Output resolution	28×28	32×32	64×64
Architecture changes	As above	Add 5th conv layer	Start with ngf*16, add upsampling layers
Data normalization	[0.5], [0.5]	[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]	Same as CIFAR

For rectangular images or higher resolutions, adjust kernel_size, stride, and padding in each ConvTranspose2d/Conv2d pair to maintain spatial dimension consistency.

---

Related Resources in AI-For-Beginners

The repository contains additional notebooks that extend these concepts:

• lessons/4-ComputerVision/10-GANs/GANTF.ipynb — TensorFlow/Keras implementation of the same DCGAN architecture for framework comparison
• lessons/4-ComputerVision/10-GANs/StyleTransfer.ipynb — PyTorch neural style transfer using pretrained VGG features
• lessons/4-ComputerVision/09-Autoencoders/AutoencodersPyTorch.ipynb — Autoencoder foundations that precede Variational Autoencoders (VAEs), an alternative generative approach

---

Summary

• GANs for image generation require two networks: a Generator that creates images from noise via transposed convolutions, and a Discriminator that classifies real versus fake via strided convolutions
• Binary cross-entropy loss drives the adversarial objective, with the generator optimizing to fool the discriminator and the discriminator optimizing to detect fakes
• Alternating updates with separate Adam optimizers (betas=(0.5, 0.999)) stabilize training and prevent mode collapse
• The lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb file in Microsoft's AI-For-Beginners repository provides a complete, educational implementation that scales from MNIST to custom datasets

---

Frequently Asked Questions

What is the difference between a Generator and Discriminator in a GAN?

The Generator is a neural network that transforms random noise vectors into synthetic images through upsampling layers like ConvTranspose2d. The Discriminator is a binary classifier that downsamples input images through Conv2d layers to output a probability of realness. They compete: the Generator tries to maximize the Discriminator's error rate, while the Discriminator tries to minimize it.

Why does the generator use real labels for fake images during training?

This is the adversarial training trick. The Generator's loss is calculated as criterion(D(G(z)), real_labels) instead of criterion(D(G(z)), fake_labels). This formulation makes the Generator's objective identical to "maximize log(D(G(z)))" from the original GAN paper, effectively training the Generator to produce outputs that the Discriminator classifies as real.

How can I prevent mode collapse in my PyTorch GAN?

Mode collapse occurs when the Generator produces limited varieties of outputs. Mitigation strategies include: using historical averaging of past generator samples, applying feature matching (matching discriminator intermediate activations), implementing mini-batch discrimination, or switching to WGAN-GP which uses a gradient penalty instead of binary cross-entropy. The repository's DCGAN baseline includes BatchNorm and careful optimizer settings as foundational stabilizers.

Can I use this GAN code for 1024×1024 images without modification?

No—significant architectural changes are required for high-resolution image generation. The NVIDIA StyleGAN architecture (and its successors) replaces the simple DCGAN design with progressive growing, style-based latent spaces, and skip connections. For 1024×1024 outputs, use established implementations like torchvision models or the official StyleGAN2/3 PyTorch ports rather than extending this educational DCGAN.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}