How to Implement Autoencoders and VAEs for Image Compression with PyTorch in AI For Beginners

You can implement convolutional autoencoders and variational autoencoders (VAEs) for image compression in PyTorch by defining encoder-decoder architectures with bottleneck layers, applying the reparameterization trick to sample latent vectors, and minimizing a composite loss function that balances reconstruction accuracy with KL divergence.

The microsoft/AI-For-Beginners repository provides a complete reference implementation for building image compression models using these architectures. This guide walks through the code in lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb, demonstrating how to implement autoencoders VAEs image compression PyTorch workflows that reduce dimensionality while preserving visual fidelity.

Understanding Autoencoders and VAEs for Image Compression

Autoencoders are neural networks that learn compressed representations of data by training an encoder to map inputs to a latent space and a decoder to reconstruct the original input from that latent representation. Variational Autoencoders (VAEs) extend this by learning probabilistic distributions over the latent space, enabling both compression and generative sampling.

In the context of image compression, the bottleneck layer (latent dimension) forces the network to learn efficient, lossy encodings. The smaller the latent dimension relative to the input size, the higher the compression ratio.

Architecture Components in AI For Beginners

The PyTorch implementation in the repository defines three core classes: VAEEncoder, VAEDecoder, and VAEAutoEncoder. These use convolutional layers optimized for MNIST and adaptable to other image formats.

The Convolutional Encoder

The VAEEncoder class processes input images through convolutional layers to extract hierarchical features, flattening them into a latent vector. For image compression, the encoder progressively reduces spatial dimensions while increasing channel depth.

import torch
import torch.nn as nn

class VAEEncoder(nn.Module):
    def __init__(self, latent_dim=2):
        super().__init__()
        # Convolutional layers reduce spatial dimensions

        self.conv = nn.Sequential(
            nn.Conv2d(1, 32, kernel_size=3, stride=2, padding=1),
            nn.ReLU(),
            nn.Conv2d(32, 64, kernel_size=3, stride=2, padding=1),
            nn.ReLU()
        )
        # Calculate flattened size (e.g., for 28x28 MNIST: 64 * 7 * 7 = 3136)

        self.fc_mu = nn.Linear(64 * 7 * 7, latent_dim)
        self.fc_logvar = nn.Linear(64 * 7 * 7, latent_dim)
    
    def forward(self, x):
        x = self.conv(x)
        x = x.view(x.size(0), -1)
        mu = self.fc_mu(x)
        logvar = self.fc_logvar(x)
        return mu, logvar

The Reparameterization Trick

VAEs use the reparameterization trick to enable backpropagation through stochastic sampling. Instead of sampling directly from the latent distribution, the model samples from a standard normal distribution and shifts/ scales it using the learned mean (mu) and log variance (logvar).

def reparameterize(self, mu, logvar):
    std = torch.exp(0.5 * logvar)
    eps = torch.randn_like(std)
    return mu + eps * std

This operation, implemented within the VAEAutoEncoder forward pass in AutoEncodersPyTorch.ipynb, ensures the latent space remains continuous and differentiable.

The Convolutional Decoder

The VAEDecoder class reconstructs images from the latent vector using transposed convolutions (deconvolutions) to upsample the representation back to the original dimensions.

class VAEDecoder(nn.Module):
    def __init__(self, latent_dim=2):
        super().__init__()
        self.fc = nn.Linear(latent_dim, 64 * 7 * 7)
        self.deconv = nn.Sequential(
            nn.ConvTranspose2d(64, 32, kernel_size=3, stride=2, padding=1, output_padding=1),
            nn.ReLU(),
            nn.ConvTranspose2d(32, 1, kernel_size=3, stride=2, padding=1, output_padding=1),
            nn.Sigmoid()  # Normalize output to [0, 1]

        )
    
    def forward(self, z):
        x = self.fc(z)
        x = x.view(x.size(0), 64, 7, 7)
        x = self.deconv(x)
        return x

Implementing the Complete VAE Model

The VAEAutoEncoder class combines these components into a single architecture that manages the encoding, reparameterization, and decoding pipeline.

class VAEAutoEncoder(nn.Module):
    def __init__(self, latent_dim=2):
        super().__init__()
        self.encoder = VAEEncoder(latent_dim)
        self.decoder = VAEDecoder(latent_dim)
    
    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        return mu + eps * std
    
    def forward(self, x):
        mu, logvar = self.encoder(x)
        z = self.reparameterize(mu, logvar)
        recon = self.decoder(z)
        return recon, mu, logvar

This implementation, as defined in lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb, returns both the reconstruction and the distribution parameters needed for the loss calculation.

Training Workflow and Loss Functions

Training a VAE for image compression requires optimizing two objectives simultaneously: reconstruction loss (typically Mean Squared Error or Binary Cross-Entropy) and the KL Divergence loss, which regularizes the latent distribution to match a standard normal prior.

def vae_loss(recon_x, x, mu, logvar):
    # Reconstruction loss

    BCE = nn.functional.binary_cross_entropy(recon_x, x, reduction='sum')
    # KL divergence: -0.5 * sum(1 + log(sigma^2) - mu^2 - sigma^2)

    KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return BCE + KLD

The training loop in the notebook iterates over batches, computing this composite loss and updating weights via standard backpropagation:

model = VAEAutoEncoder(latent_dim=2).to(device)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

for epoch in range(epochs):
    for batch_idx, (data, _) in enumerate(train_loader):
        data = data.to(device)
        optimizer.zero_grad()
        recon_batch, mu, logvar = model(data)
        loss = vae_loss(recon_batch, data, mu, logvar)
        loss.backward()
        optimizer.step()

Controlling Compression Ratio with Latent Dimensions

The compression ratio is determined by the latent dimension parameter passed to the encoder and decoder. For a 28×28 MNIST image (784 pixels), a latent_dim of 2 represents a compression ratio of 392:1, while a dimension of 32 achieves a 24.5:1 ratio.

Adjust this parameter based on your reconstruction quality requirements versus storage constraints. The repository demonstrates visualizations comparing different latent dimensions in AutoEncodersPyTorch.ipynb.

Adapting Code for Custom Datasets

While the repository examples target MNIST, you can adapt the architecture for CIFAR-10 or custom grayscale images by modifying three components:

• Input channels: Change nn.Conv2d(1, ...) to nn.Conv2d(3, ...) for RGB images
• Spatial calculations: Update the flattened size calculation (e.g., 64 * 7 * 7 becomes 64 * 8 * 8 for 32×32 CIFAR-10 inputs with the same stride pattern)
• Latent dimension: Increase latent_dim for higher-resolution images to maintain reconstruction quality

The TensorFlow implementation in lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb provides an alternative reference for these architectural modifications using Keras layers.

Summary

• Autoencoders compress images by learning encoder-decoder mappings through a bottleneck layer, while VAEs add probabilistic latent spaces for generative capabilities.
• The VAEEncoder, VAEDecoder, and VAEAutoEncoder classes in AutoEncodersPyTorch.ipynb provide a complete PyTorch implementation for image compression tasks.
• Reparameterization enables differentiable sampling from the latent distribution, critical for VAE training.
• The loss function combines reconstruction error (BCE/MSE) with KL divergence to balance fidelity and regularization.
• Compression ratio scales inversely with latent dimension size, configurable via the latent_dim parameter.
• Architectures adapt to custom datasets by adjusting input channels and spatial dimensions in the convolutional layers.

Frequently Asked Questions

What is the difference between a standard autoencoder and a VAE?

A standard autoencoder learns deterministic mappings to a latent vector, while a VAE learns parameters of a probability distribution (mean and variance) for each input. According to the microsoft/AI-For-Beginners implementation, VAEs use the reparameterization trick to sample from this distribution, making the latent space continuous and suitable for generating new images, whereas standard autoencoders produce discrete encodings optimized purely for reconstruction.

How do I choose the right latent dimension for image compression?

Select a latent dimension that balances compression ratio against reconstruction quality. For MNIST (28×28), dimensions between 2 and 32 work well; for higher-resolution images like CIFAR-10 (32×32), use 64–128 dimensions. Monitor the validation loss in AutoEncodersPyTorch.ipynb—if reconstructions are too blurry, increase the latent dimension; if file size reduction is insufficient, decrease it.

Can I use these models for color images (RGB)?

Yes, but you must modify the input channel dimension in the encoder's first convolutional layer from 1 to 3, and adjust the decoder's final layer to output 3 channels. The repository's architecture uses nn.Conv2d(1, 32, ...) for grayscale; changing this to nn.Conv2d(3, 32, ...) enables processing of RGB datasets like CIFAR-10 or custom color images.

Why is my VAE producing blurry reconstructions?

Blurry outputs typically indicate excessive compression (latent dimension too small) or insufficient training epochs. The VAE loss function in the repository combines reconstruction loss with KL divergence weighting—try reducing the relative weight of the KL term or increasing the latent dimension to allow more information through the bottleneck. Also verify that your decoder uses nn.Sigmoid() activation to constrain pixel values to [0, 1].

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}