# How to Build Generative Adversarial Networks (GANs) for Image Generation Using PyTorch > Learn to build Generative Adversarial Networks GANs for image generation with PyTorch. Define Generator and Discriminator networks, train them using Adam optimizers and binary cross-entropy loss. - Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners) - Tags: tutorial - Published: 2026-08-08 --- **Build a DCGAN in PyTorch by defining a Generator network that upsamples noise vectors into images and a Discriminator network that classifies real versus fake, then train both with alternating Adam optimizers using binary cross-entropy loss.** A **Generative Adversarial Network (GAN)** synthesizes realistic images by pitting two neural networks against each other in a zero-sum game. The **AI-For-Beginners** repository by Microsoft provides a complete, beginner-friendly implementation that demonstrates this architecture on the MNIST dataset. This guide walks through the exact code structure, training dynamics, and practical adaptations you need to build GANs for image generation using PyTorch. --- ## GAN Architecture Components Every GAN consists of two competing networks. According to the source code in `lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb`, these components are implemented as separate `nn.Module` classes. ### Generator Network The **Generator** maps a random latent vector **z** into a synthetic image. In the repository's implementation (around line 103), this uses transposed convolutions to progressively upsample the tensor: ```python class Generator(nn.Module): def __init__(self, nz=100, ngf=64, nc=1): super(Generator, self).__init__() self.main = nn.Sequential( # input is Z, go through a transposed conv nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False), nn.BatchNorm2d(ngf * 8), nn.ReLU(True), nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 2, 1, bias=False), nn.BatchNorm2d(ngf * 4), nn.ReLU(True), nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False), nn.BatchNorm2d(ngf * 2), nn.ReLU(True), nn.ConvTranspose2d(ngf * 2, nc, 4, 2, 1, bias=False), nn.Tanh() # output in [-1, 1] ) ``` Key architectural decisions in this generator: - **Input:** 100-dimensional noise vector (`nz = 100`), reshaped to `(batch, 100, 1, 1)` - **Upsampling path:** Four `ConvTranspose2d` layers expand 1×1 → 4×4 → 7×7 → 14×14 → 28×28 - **BatchNorm2d:** Stabilizes training by normalizing intermediate feature maps - **Tanh output:** Constrains generated pixels to `[-1, 1]`, matching normalized image data ### Discriminator Network The **Discriminator** classifies images as real or fake. The implementation (around line 163) uses strided convolutions to downsample: ```python class Discriminator(nn.Module): def __init__(self, ndf=64, nc=1): super(Discriminator, self).__init__() self.main = nn.Sequential( # input is (nc) x 28 x 28 nn.Conv2d(nc, ndf, 4, 2, 1, bias=False), nn.LeakyReLU(0.2, inplace=True), nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 2), nn.LeakyReLU(0.2, inplace=True), nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 4), nn.LeakyReLU(0.2, inplace=True), nn.Conv2d(ndf * 4, 1, 4, 1, 0, bias=False), nn.Sigmoid() # probability of being real ) ``` Critical discriminator design choices: - **LeakyReLU(0.2):** Allows small negative gradients, preventing dead neurons - **No BatchNorm on first layer:** Preserves raw input statistics for more stable training - **Sigmoid output:** Produces probability ∈ [0, 1] for binary classification --- ## Loss Functions and Training Objectives The GAN training objective is a **binary cross-entropy (BCE)** minimax game. Both networks share the same loss function but optimize opposing goals: ```python criterion = nn.BCELoss() ``` | Network | Objective | Mathematical Form | |---------|-----------|-------------------| | **Discriminator** | Maximize `log(D(x)) + log(1 - D(G(z)))` | Correctly classify real and fake | | **Generator** | Maximize `log(D(G(z)))` | Fool discriminator into labeling fakes as real | The generator's loss uses **real labels** for its fake outputs—this is the "adversarial" trick that trains `G` to produce convincing forgeries. --- ## Implementing the GAN Training Loop in PyTorch The training loop (lines 200-300 in `GANPyTorch.ipynb`) alternates between updating the discriminator and generator. This separation prevents one network from dominating too early: ```python for epoch in range(epochs): for real_imgs, _ in dataloader: # Move to GPU if available real_imgs = real_imgs.to(device) batch_size = real_imgs.size(0) # Create labels real_labels = torch.full((batch_size, 1), 1., device=device) fake_labels = torch.full((batch_size, 1), 0., device=device) # ------------------------------------------------- # Train Discriminator: maximize log(D(x)) + log(1-D(G(z))) # ------------------------------------------------- # Forward pass with real images output_real = netD(real_imgs) loss_real = criterion(output_real, real_labels) # Generate fake images (detached from generator graph) noise = torch.randn(batch_size, nz, 1, 1, device=device) fake_imgs = netG(noise) # Forward pass with fake images output_fake = netD(fake_imgs.detach()) loss_fake = criterion(output_fake, fake_labels) # Combined discriminator loss and update d_loss = loss_real + loss_fake optimizerD.zero_grad() d_loss.backward() optimizerD.step() # ------------------------------------------------- # Train Generator: maximize log(D(G(z))) # ------------------------------------------------- # Generator wants discriminator to label fakes as real output = netD(fake_imgs) g_loss = criterion(output, real_labels) # Note: real_labels! optimizerG.zero_grad() g_loss.backward() optimizerG.step() ``` Critical training mechanics: - **`.detach()`:** Prevents gradients from flowing into the generator during discriminator updates - **Separate optimizers:** `optimizerD` and `optimizerG` each manage one network's parameters - **Adam with β₁=0.5:** The repository uses `betas=(0.5, 0.999)` to handle unstable GAN gradients --- ## Complete Standalone GAN Implementation Below is a minimal, runnable script adapted from `GANPyTorch.ipynb` that you can execute on any image dataset. The architecture matches the repository's DCGAN implementation: ```python # ------------------------------------------------- # Minimal DCGAN for image generation - PyTorch # ------------------------------------------------- import torch import torch.nn as nn import torch.optim as optim from torchvision import datasets, transforms from torch.utils.data import DataLoader # ------------------- Hyperparameters ------------------- device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') nz = 100 # latent dimension ngf = 64 # generator feature maps ndf = 64 # discriminator feature maps nc = 1 # image channels (1 for MNIST, 3 for RGB) batch_size = 128 lr = 2e-4 epochs = 50 # ------------------- Data Loading ---------------------- transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize([0.5], [0.5]) # scales to [-1, 1] ]) train_loader = DataLoader( datasets.MNIST(root='./data', train=True, download=True, transform=transform), batch_size=batch_size, shuffle=True, num_workers=2 ) # ------------------- Generator ------------------------- class Generator(nn.Module): def __init__(self): super().__init__() self.main = nn.Sequential( # input: (nz, 1, 1) nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False), nn.BatchNorm2d(ngf * 8), nn.ReLU(True), # state: (ngf*8, 4, 4) nn.ConvTranspose2d(ngf * 8, ngf * 4, 3, 2, 1, bias=False), nn.BatchNorm2d(ngf * 4), nn.ReLU(True), # state: (ngf*4, 7, 7) nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False), nn.BatchNorm2d(ngf * 2), nn.ReLU(True), # state: (ngf*2, 14, 14) nn.ConvTranspose2d(ngf * 2, nc, 4, 2, 1, bias=False), nn.Tanh() # output: (nc, 28, 28) ) def forward(self, z): return self.main(z) # ------------------- Discriminator --------------------- class Discriminator(nn.Module): def __init__(self): super().__init__() self.main = nn.Sequential( # input: (nc, 28, 28) nn.Conv2d(nc, ndf, 4, 2, 1, bias=False), nn.LeakyReLU(0.2, inplace=True), # state: (ndf, 14, 14) nn.Conv2d(ndf, ndf * 2, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 2), nn.LeakyReLU(0.2, inplace=True), # state: (ndf*2, 7, 7) nn.Conv2d(ndf * 2, ndf * 4, 4, 2, 1, bias=False), nn.BatchNorm2d(ndf * 4), nn.LeakyReLU(0.2, inplace=True), # state: (ndf*4, 3, 3) nn.Conv2d(ndf * 4, 1, 4, 1, 0, bias=False), nn.Sigmoid() # output: (1, 1, 1) -> scalar probability ) def forward(self, x): return self.main(x) # ------------------- Initialization -------------------- netG = Generator().to(device) netD = Discriminator().to(device) # Apply custom weight initialization (as in DCGAN paper) def weights_init(m): classname = m.__class__.__name__ if classname.find('Conv') != -1: nn.init.normal_(m.weight.data, 0.0, 0.02) elif classname.find('BatchNorm') != -1: nn.init.normal_(m.weight.data, 1.0, 0.02) nn.init.constant_(m.bias.data, 0) netG.apply(weights_init) netD.apply(weights_init) # Loss and optimizers criterion = nn.BCELoss() optimizerD = optim.Adam(netD.parameters(), lr=lr, betas=(0.5, 0.999)) optimizerG = optim.Adam(netG.parameters(), lr=lr, betas=(0.5, 0.999)) # ------------------- Training Loop --------------------- for epoch in range(epochs): for i, (real_imgs, _) in enumerate(train_loader): real_imgs = real_imgs.to(device) cur_batch_size = real_imgs.size(0) real_labels = torch.full((cur_batch_size, 1, 1, 1), 1., device=device) fake_labels = torch.full((cur_batch_size, 1, 1, 1), 0., device=device) # ------- Update Discriminator ------- # Real batch netD.zero_grad() output_real = netD(real_imgs) loss_real = criterion(output_real, real_labels) # Fake batch noise = torch.randn(cur_batch_size, nz, 1, 1, device=device) fake_imgs = netG(noise) output_fake = netD(fake_imgs.detach()) loss_fake = criterion(output_fake, fake_labels) # Combined and backward loss_D = loss_real + loss_fake loss_D.backward() optimizerD.step() # ------- Update Generator ------- netG.zero_grad() output = netD(fake_imgs) loss_G = criterion(output, real_labels) loss_G.backward() optimizerG.step() # Print progress if i % 100 == 0: print(f'Epoch [{epoch}/{epochs}] Batch [{i}/{len(train_loader)}] ' f'Loss_D: {loss_D.item():.4f} Loss_G: {loss_G.item():.4f}') print("Training complete!") ``` --- ## Adapting to Different Image Datasets To build GANs for image generation beyond MNIST, modify these parameters: | Parameter | MNIST | CIFAR-10 | CelebA (64×64) | |-----------|-------|----------|----------------| | `nc` (channels) | 1 | 3 | 3 | | Output resolution | 28×28 | 32×32 | 64×64 | | Architecture changes | As above | Add 5th conv layer | Start with `ngf*16`, add upsampling layers | | Data normalization | `[0.5], [0.5]` | `[0.5, 0.5, 0.5], [0.5, 0.5, 0.5]` | Same as CIFAR | For rectangular images or higher resolutions, adjust `kernel_size`, `stride`, and `padding` in each `ConvTranspose2d`/`Conv2d` pair to maintain spatial dimension consistency. --- ## Related Resources in AI-For-Beginners The repository contains additional notebooks that extend these concepts: - **`lessons/4-ComputerVision/10-GANs/GANTF.ipynb`** — TensorFlow/Keras implementation of the same DCGAN architecture for framework comparison - **`lessons/4-ComputerVision/10-GANs/StyleTransfer.ipynb`** — PyTorch neural style transfer using pretrained VGG features - **`lessons/4-ComputerVision/09-Autoencoders/AutoencodersPyTorch.ipynb`** — Autoencoder foundations that precede Variational Autoencoders (VAEs), an alternative generative approach --- ## Summary - **GANs for image generation** require two networks: a **Generator** that creates images from noise via transposed convolutions, and a **Discriminator** that classifies real versus fake via strided convolutions - **Binary cross-entropy loss** drives the adversarial objective, with the generator optimizing to fool the discriminator and the discriminator optimizing to detect fakes - **Alternating updates** with separate Adam optimizers (`betas=(0.5, 0.999)`) stabilize training and prevent mode collapse - The `lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb` file in Microsoft's **AI-For-Beginners** repository provides a complete, educational implementation that scales from MNIST to custom datasets --- ## Frequently Asked Questions ### What is the difference between a Generator and Discriminator in a GAN? The **Generator** is a neural network that transforms random noise vectors into synthetic images through upsampling layers like `ConvTranspose2d`. The **Discriminator** is a binary classifier that downsamples input images through `Conv2d` layers to output a probability of realness. They compete: the Generator tries to maximize the Discriminator's error rate, while the Discriminator tries to minimize it. ### Why does the generator use real labels for fake images during training? This is the **adversarial training trick**. The Generator's loss is calculated as `criterion(D(G(z)), real_labels)` instead of `criterion(D(G(z)), fake_labels)`. This formulation makes the Generator's objective identical to "maximize log(D(G(z)))" from the original GAN paper, effectively training the Generator to produce outputs that the Discriminator classifies as real. ### How can I prevent mode collapse in my PyTorch GAN? **Mode collapse** occurs when the Generator produces limited varieties of outputs. Mitigation strategies include: using **historical averaging** of past generator samples, applying **feature matching** (matching discriminator intermediate activations), implementing **mini-batch discrimination**, or switching to **WGAN-GP** which uses a gradient penalty instead of binary cross-entropy. The repository's DCGAN baseline includes BatchNorm and careful optimizer settings as foundational stabilizers. ### Can I use this GAN code for 1024×1024 images without modification? No—significant architectural changes are required for high-resolution image generation. The NVIDIA **StyleGAN** architecture (and its successors) replaces the simple DCGAN design with **progressive growing**, **style-based latent spaces**, and **skip connections**. For 1024×1024 outputs, use established implementations like `torchvision` models or the official StyleGAN2/3 PyTorch ports rather than extending this educational DCGAN.