# Implementing Generative Adversarial Networks (GANs) for Image Synthesis: A Complete Guide with TensorFlow and PyTorch

> Learn to implement generative adversarial networks GANs for image synthesis. Build a DCGAN with TensorFlow and PyTorch using runnable code examples from AI-For-Beginners.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: how-to-guide
- Published: 2026-08-10

---

**Build a working DCGAN for image generation using the Microsoft AI-For-Beginners curriculum with complete, runnable code examples in both TensorFlow/Keras and PyTorch.**

The **AI-For-Beginners** repository by Microsoft provides a comprehensive, beginner-friendly introduction to **generative adversarial networks (GANs) for image synthesis**. This guide walks through the exact implementation patterns found in the official lesson materials, covering architecture design, the adversarial training loop, and practical code you can run immediately.

## What Are GANs and How Do They Generate Images?

A **Generative Adversarial Network** consists of two neural networks locked in a minimax game: a **Generator** that creates synthetic images from random noise, and a **Discriminator** that learns to distinguish real images from fakes. Through adversarial training, the Generator progressively improves its ability to synthesize photorealistic images.

According to the AI-For-Beginners source code, the key educational resources are:

- Theory and architecture documentation: [[`lessons/4-ComputerVision/10-GANs/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/README.md)](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/README.md)
- TensorFlow/Keras implementation: [`lessons/4-ComputerVision/10-GANs/GANTF.ipynb`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/GANTF.ipynb)
- PyTorch implementation: [`lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb)

## DCGAN Architecture for Image Synthesis

The **Deep Convolutional GAN (DCGAN)** is the standard architecture for image generation tasks. The Microsoft curriculum implements this pattern with specific layer configurations optimized for 28×28 MNIST images.

### Generator Network

The **Generator** maps a latent vector **z** (typically 100-dimensional) to a synthetic image through transposed convolutions:

| Stage | Operation | Output Shape |
|-------|-----------|--------------|
| Input | Dense projection + reshape | 7×7×256 |
| Block 1 | Conv2DTranspose (128 filters) | 7×7×128 |
| Block 2 | Conv2DTranspose (64 filters, stride 2) | 14×14×64 |
| Output | Conv2DTranspose (1 channel, stride 2) + Tanh | 28×28×1 |

### Discriminator Network

The **Discriminator** acts as a binary classifier with a mirrored convolutional structure:

| Stage | Operation | Output Shape |
|-------|-----------|--------------|
| Input | 28×28×1 image | 28×28×1 |
| Block 1 | Conv2D (64 filters, stride 2) + LeakyReLU | 14×14×64 |
| Block 2 | Conv2D (128 filters, stride 2) + LeakyReLU | 7×7×128 |
| Output | Flatten + Dense + Sigmoid | Scalar probability |

## TensorFlow/Keras Implementation of GANs for Image Synthesis

The following implementation matches the official `GANTF.ipynb` notebook structure, using functional and sequential APIs for clarity:

```python
import tensorflow as tf
from tensorflow.keras import layers, models, optimizers
import numpy as np

# Configuration

latent_dim = 100
img_shape = (28, 28, 1)  # MNIST grayscale format

def build_generator():
    """Builds the generator network as defined in AI-For-Beginners."""
    model = models.Sequential(name="Generator")
    
    # Foundation: dense projection to spatial form

    model.add(layers.Dense(7 * 7 * 256, use_bias=False, input_shape=(latent_dim,)))
    model.add(layers.BatchNormalization())
    model.add(layers.ReLU())
    model.add(layers.Reshape((7, 7, 256)))
    
    # Upsampling block 1: 7×7 → 7×7 (feature expansion)

    model.add(layers.Conv2DTranspose(128, 5, strides=1, padding='same', use_bias=False))
    model.add(layers.BatchNormalization())
    model.add(layers.ReLU())
    
    # Upsampling block 2: 7×7 → 14×14

    model.add(layers.Conv2DTranspose(64, 5, strides=2, padding='same', use_bias=False))
    model.add(layers.BatchNormalization())
    model.add(layers.ReLU())
    
    # Output: 14×14 → 28×28 with tanh activation

    model.add(layers.Conv2DTranspose(1, 5, strides=2, padding='same',
                                     activation='tanh', use_bias=False))
    return model

def build_discriminator():
    """Builds the discriminator network with leaky activations."""
    model = models.Sequential(name="Discriminator")
    
    # Downsampling block 1

    model.add(layers.Conv2D(64, 5, strides=2, padding='same',
                            input_shape=img_shape))
    model.add(layers.LeakyReLU(alpha=0.2))
    model.add(layers.Dropout(0.3))
    
    # Downsampling block 2

    model.add(layers.Conv2D(128, 5, strides=2, padding='same'))
    model.add(layers.LeakyReLU(alpha=0.2))
    model.add(layers.Dropout(0.3))
    
    # Classification head

    model.add(layers.Flatten())
    model.add(layers.Dense(1, activation='sigmoid'))
    return model

# Instantiate networks

generator = build_generator()
discriminator = build_discriminator()
discriminator.compile(loss='binary_crossentropy',
                      optimizer=optimizers.Adam(1e-4))

# Combined GAN: generator → discriminator (frozen during generator training)

discriminator.trainable = False
gan_input = layers.Input(shape=(latent_dim,))
gan_output = discriminator(generator(gan_input))
gan = models.Model(gan_input, gan_output, name="GAN")
gan.compile(loss='binary_crossentropy', optimizer=optimizers.Adam(1e-4))

```

### Training Loop for TensorFlow GAN

The adversarial training procedure alternates between discriminator and generator updates:

```python

# Load and preprocess MNIST

(train_images, _) = tf.keras.datasets.mnist.load_data()
train_images = (train_images.astype('float32') - 127.5) / 127.5
train_images = np.expand_dims(train_images, axis=-1)

batch_size, epochs = 64, 10000
half_batch = batch_size // 2

for epoch in range(epochs):
    # --- Discriminator training ---

    # Sample real images

    idx = np.random.randint(0, train_images.shape[0], half_batch)
    real_imgs = train_images[idx]
    
    # Generate fake images from random noise

    noise = np.random.normal(0, 1, (half_batch, latent_dim))
    fake_imgs = generator.predict(noise, verbose=0)
    
    # Train discriminator on both classes

    d_loss_real = discriminator.train_on_batch(real_imgs, np.ones((half_batch, 1)))
    d_loss_fake = discriminator.train_on_batch(fake_imgs, np.zeros((half_batch, 1)))
    d_loss = 0.5 * (d_loss_real + d_loss_fake)
    
    # --- Generator training ---

    # Generate fresh noise samples

    noise = np.random.normal(0, 1, (batch_size, latent_dim))
    
    # Train generator to fool discriminator (target = 1 for "real")

    g_loss = gan.train_on_batch(noise, np.ones((batch_size, 1)))
    
    if epoch % 1000 == 0:
        print(f'Epoch {epoch:5d} | D loss: {d_loss:.4f} | G loss: {g_loss:.4f}')

```

## PyTorch Implementation of GANs for Image Synthesis

The PyTorch version in `GANPyTorch.ipynb` uses equivalent operations with explicit manual training steps:

```python
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms

latent_dim = 100
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

class Generator(nn.Module):
    """PyTorch generator with ConvTranspose2d upsampling."""
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            # Project and reshape

            nn.Linear(latent_dim, 256 * 7 * 7),
            nn.BatchNorm1d(256 * 7 * 7),
            nn.ReLU(True),
            nn.Unflatten(1, (256, 7, 7)),
            
            # 7×7 → 14×14

            nn.ConvTranspose2d(256, 128, 4, stride=2, padding=1),
            nn.BatchNorm2d(128),
            nn.ReLU(True),
            
            # 14×14 → 28×28 (intermediate)

            nn.ConvTranspose2d(128, 64, 4, stride=2, padding=1),
            nn.BatchNorm2d(64),
            nn.ReLU(True),
            
            # Final: slightly adjust to exact 28×28

            nn.ConvTranspose2d(64, 1, 4, stride=2, padding=1),
            nn.Tanh()  # Output range [-1, 1]

        )
    
    def forward(self, z):
        return self.net(z)

class Discriminator(nn.Module):
    """PyTorch discriminator with LeakyReLU activations."""
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv2d(1, 64, 4, stride=2, padding=1),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Dropout2d(0.3),
            
            nn.Conv2d(64, 128, 4, stride=2, padding=1),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Dropout2d(0.3),
            
            nn.Flatten(),
            nn.Linear(128 * 7 * 7, 1),
            nn.Sigmoid()
        )
    
    def forward(self, img):
        return self.net(img)

# Instantiate and move to device

generator = Generator().to(device)
discriminator = Discriminator().to(device)

criterion = nn.BCELoss()
opt_G = optim.Adam(generator.parameters(), lr=2e-4)
opt_D = optim.Adam(discriminator.parameters(), lr=2e-4)

```

### Training Loop for PyTorch GAN

Note the explicit gradient handling with `zero_grad()`, `backward()`, and `step()`:

```python

# Data loading

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize([0.5], [0.5])
])

dataloader = torch.utils.data.DataLoader(
    datasets.MNIST('.', train=True, download=True, transform=transform),
    batch_size=64, shuffle=True
)

for epoch in range(10000):
    for real_imgs, _ in dataloader:
        real_imgs = real_imgs.to(device)
        batch_size = real_imgs.size(0)
        
        real_labels = torch.ones(batch_size, 1, device=device)
        fake_labels = torch.zeros(batch_size, 1, device=device)
        
        # ----- Train Discriminator -----

        opt_D.zero_grad()
        
        # Loss on real images

        outputs_real = discriminator(real_imgs)
        d_loss_real = criterion(outputs_real, real_labels)
        
        # Loss on fake images (detach prevents generator gradients)

        z = torch.randn(batch_size, latent_dim, device=device)
        fake_imgs = generator(z)
        outputs_fake = discriminator(fake_imgs.detach())
        d_loss_fake = criterion(outputs_fake, fake_labels)
        
        d_loss = (d_loss_real + d_loss_fake) / 2
        d_loss.backward()
        opt_D.step()
        
        # ----- Train Generator -----

        opt_G.zero_grad()
        
        # Generator wants discriminator to classify fakes as real

        outputs_g = discriminator(fake_imgs)  # No detach() — train generator

        g_loss = criterion(outputs_g, real_labels)
        
        g_loss.backward()
        opt_G.step()
    
    if epoch % 1000 == 0:
        print(f'Epoch {epoch:5d} | D loss: {d_loss.item():.4f} | G loss: {g_loss.item():.4f}')

```

## Critical Implementation Details

### The Generator's Fooling Objective

Both frameworks implement the same mathematical objective: the **Generator minimizes log(1 − D(G(z)))** by training against labels of 1 (real). In practice, this is equivalent to maximizing **log D(G(z))** — the probability that the discriminator assigns to generated images.

### Discriminator Weight Freezing in TensorFlow

The line `discriminator.trainable = False` before compiling the combined GAN is essential. It ensures that when `gan.train_on_batch()` is called, gradients flow only through the generator's weights.

### Detach() in PyTorch

The `.detach()` method creates a tensor without gradient history. This is applied to `fake_imgs` during discriminator training to prevent generator weights from receiving gradients in that phase.

## Common Pitfalls in GAN Image Synthesis

Based on the AI-For-Beginners curriculum and established best practices, watch for these issues:

| Problem | Diagnostic | Solution |
|---------|-----------|----------|
| **Mode collapse** | Generator outputs identical images regardless of input noise | Implement **mini-batch discrimination** or **feature matching** loss |
| **Vanishing gradients** | Discriminator loss drops to 0, generator loss plateaus | Switch to **WGAN-GP** loss with gradient penalty for more stable training dynamics |
| **Training imbalance** | One network dominates immediately | Use **TTUR** (Two Time-Scale Update Rule): set generator learning rate higher than discriminator |
| **Checkerboard artifacts** | Grid-like patterns in generated images | Use stride-2 convolutions with appropriate kernel sizes; avoid mismatching stride and kernel |
| **High-resolution instability** | Training diverges above 64×64 resolution | Apply **progressive growing**: start at low resolution, gradually add layers |

## Extending Your GAN Implementation

The foundation in `GANTF.ipynb` and `GANPyTorch.ipynb` supports several advanced directions:

- **Conditional GANs**: Concatenate one-hot class labels to the latent vector for controlled generation of specific digits
- **CIFAR-10/Custom datasets**: Modify `img_shape` to (32, 32, 3) and adjust generator/discriminator depths
- **Style-based architectures**: Replace the single latent vector with a learned constant and adaptive instance normalization (StyleGAN approach)
- **Wasserstein loss**: Replace `BinaryCrossentropy` with a critic-based loss and weight clipping or gradient penalty

The AI-For-Beginners GAN lesson also covers **neural style transfer** using pretrained VGG features for content and style loss computation — see the README for architectural details.

## Summary

Implementing **generative adversarial networks for image synthesis** using the Microsoft AI-For-Beginners repository provides:

- Complete **DCGAN architectures** in both TensorFlow/Keras and PyTorch with matching layer configurations
- Explicit **adversarial training loops** that alternate discriminator and generator updates while managing gradient flow
- **Practical MNIST examples** that generate recognizable digits within thousands of iterations
- Clear pathways to **advanced techniques** including conditional generation, WGAN-GP, and progressive growing

The provided notebooks at `lessons/4-ComputerVision/10-GANs/` contain the full implementations with additional visualization code for monitoring training progress.

## Frequently Asked Questions

### What is the difference between a standard GAN and a DCGAN?

A **DCGAN (Deep Convolutional GAN)** replaces the fully-connected layers in early GAN implementations with **strided convolutions** in the discriminator and **transposed convolutions** in the generator. As implemented in the AI-For-Beginners curriculum, this architectural change stabilizes training and produces higher-quality images by leveraging spatial structure through convolutional operations.

### Why does the generator use tanh activation for output?

The **tanh activation** constrains pixel values to the range [-1, 1], matching the normalized input distribution. The training code applies `(images - 127.5) / 127.5` to scale MNIST from [0, 255] to [-1, 1]. This symmetric range centers the data and improves gradient flow compared to sigmoid's [0, 1] range.

### How long does it take to train a functional GAN on MNIST?

With the hyperparameters from `GANTF.ipynb` — batch size 64, learning rate 1e-4, on modern hardware — **coherent digit generation typically emerges within 5-10 epochs** (a few minutes on GPU). High-quality, diverse outputs require 50-100+ epochs. The curriculum recommends monitoring both losses: healthy training shows oscillating discriminator loss around 0.5 and gradually decreasing or stable generator loss.

### Can these implementations scale to high-resolution image generation?

The basic DCGAN architecture struggles beyond 64×64 resolution due to training instability. For high-resolution **GAN image synthesis**, the Microsoft curriculum suggests **progressive growing** techniques where both networks start at 4×4 resolution and layers are incrementally added. This approach, combined with WGAN-GP loss modifications, enables generation of 1024×1024 images as demonstrated in contemporary research.