How to Implement Autoencoders and Variational Autoencoders (VAEs) for Image Compression

Autoencoders compress images into low-dimensional latent representations and reconstruct them, while Variational Autoencoders (VAEs) add probabilistic structure to enable generative sampling—both implemented in the microsoft/AI-For-Beginners repository using TensorFlow/Keras and PyTorch.

The microsoft/AI-For-Beginners repository provides a comprehensive Computer Vision module to implement autoencoders and Variational Autoencoders (VAEs) for image compression. These neural network architectures learn efficient data representations by encoding images into compressed latent spaces and decoding them back to original dimensions. Whether you use TensorFlow/Keras or PyTorch, the repository offers complete notebook implementations that demonstrate the full compression-reconstruction pipeline.

Understanding Autoencoders for Image Compression

An autoencoder is a neural network that learns to compress an input image into a low-dimensional latent representation and then reconstruct the image from that representation. This compression-reconstruction pipeline makes autoencoders a natural tool for image compression tasks.

According to the repository's README at lessons/4-ComputerVision/09-Autoencoders/README.md, the architecture consists of two main components:

• Encoder: A stack of Conv2D or Dense layers that reduces spatial dimensions to create a compressed latent vector
• Decoder: Mirrored Conv2DTranspose or Dense layers that up-sample the latent vector back to the original image dimensions

The model is trained to minimize reconstruction loss, typically binary_crossentropy for binary images or mse (mean squared error) for continuous pixel values.

The Compression Pipeline

To compress images using a standard autoencoder:

1. Train the model with autoencoder.fit(x_train, x_train, epochs=…) using identical input and target data
2. Extract compressed representations via latent = encoder.predict(x)
3. Reconstruct images using recon = decoder.predict(latent)

Variational Autoencoders (VAEs) Explained

A Variational Autoencoder extends the vanilla autoencoder by imposing a probabilistic structure on the latent space. Instead of learning a single deterministic code, a VAE learns a distribution (usually a Gaussian) from which latent vectors are sampled. This regularization encourages smooth latent spaces and enables generative sampling of new images.

The key differences implemented in the repository notebooks include:

• Parameterization of the latent distribution using two parallel dense layers producing mean (μ) and log-variance (logσ²)
• The reparameterization trick: z = μ + ε * exp(logσ² / 2), where ε ~ N(0, I)
• Combined loss function that adds KL divergence to the reconstruction error, encouraging the learned distribution to stay close to a unit Gaussian

Latent Space Regularization

The KL divergence term in the loss function prevents overfitting and ensures the latent space follows a standard normal distribution. This allows the VAE to generate new images by sampling random vectors from N(0, I) and passing them through the decoder.

TensorFlow and Keras Implementation

The notebook lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb provides complete TensorFlow implementations for both architectures.

Simple Autoencoder in Keras

Below is a minimal implementation for MNIST compression using Conv2D and Conv2DTranspose layers:

import tensorflow as tf
from tensorflow.keras.layers import Input, Conv2D, Conv2DTranspose, Flatten, Dense, Reshape
from tensorflow.keras.models import Model

# Load MNIST (28×28 grayscale)

(x_train, _), (x_test, _) = tf.keras.datasets.mnist.load_data()
x_train = x_train.astype("float32") / 255.0
x_test  = x_test.astype("float32") / 255.0
x_train = x_train[..., None]      # shape: (N, 28, 28, 1)

x_test  = x_test[..., None]

# Encoder

input_img = Input(shape=(28, 28, 1))
x = Conv2D(32, 3, activation="relu", strides=2, padding="same")(input_img)   # 14×14×32

x = Conv2D(64, 3, activation="relu", strides=2, padding="same")(x)          # 7×7×64

x = Flatten()(x)
latent = Dense(16, name="latent")(x)                                          # 16‑dim code

# Decoder

x = Dense(7 * 7 * 64, activation="relu")(latent)
x = Reshape((7, 7, 64))(x)
x = Conv2DTranspose(32, 3, strides=2, activation="relu", padding="same")(x) # 14×14×32

decoded = Conv2DTranspose(1, 3, strides=2, activation="sigmoid", padding="same")(x) # 28×28×1

autoencoder = Model(input_img, decoded, name="autoencoder")
autoencoder.compile(optimizer="adam", loss="binary_crossentropy")
autoencoder.summary()

# Train

autoencoder.fit(x_train, x_train, epochs=30, batch_size=128, validation_data=(x_test, x_test))

Variational Autoencoder with Reparameterization

The VAE implementation uses the functional API to handle the sampling operation and custom loss calculation:

import tensorflow as tf
from tensorflow.keras import backend as K
from tensorflow.keras.layers import Input, Conv2D, Flatten, Dense, Reshape, Conv2DTranspose, Lambda
from tensorflow.keras.models import Model

latent_dim = 2  # 2‑D latent space for easy visualization

# Encoder – outputs mean and log‑variance

inputs = Input(shape=(28, 28, 1))
x = Conv2D(32, 3, activation="relu", strides=2, padding="same")(inputs)   # 14×14×32

x = Conv2D(64, 3, activation="relu", strides=2, padding="same")(x)       # 7×7×64

x = Flatten()(x)
x = Dense(16, activation="relu")(x)
z_mean = Dense(latent_dim, name="z_mean")(x)
z_log_var = Dense(latent_dim, name="z_log_var")(x)

# Reparameterization trick

def sampling(args):
    z_mean, z_log_var = args
    epsilon = K.random_normal(shape=K.shape(z_mean))
    return z_mean + K.exp(0.5 * z_log_var) * epsilon

z = Lambda(sampling, output_shape=(latent_dim,))([z_mean, z_log_var])

# Decoder

decoder_input = Input(shape=(latent_dim,))
x = Dense(7 * 7 * 64, activation="relu")(decoder_input)
x = Reshape((7, 7, 64))(x)
x = Conv2DTranspose(32, 3, strides=2, activation="relu", padding="same")(x)  # 14×14×32

outputs = Conv2DTranspose(1, 3, strides=2, activation="sigmoid", padding="same")(x)  # 28×28×1

decoder = Model(decoder_input, outputs, name="decoder")

# VAE model

outputs = decoder(z)
vae = Model(inputs, outputs, name="vae")

# VAE loss = reconstruction + KL divergence

reconstruction_loss = tf.keras.losses.binary_crossentropy(K.flatten(inputs), K.flatten(outputs))
reconstruction_loss *= 28 * 28
kl_loss = 1 + z_log_var - K.square(z_mean) - K.exp(z_log_var)
kl_loss = -0.5 * K.sum(kl_loss, axis=-1)
vae_loss = K.mean(reconstruction_loss + kl_loss)
vae.add_loss(vae_loss)
vae.compile(optimizer="adam")
vae.summary()

# Train

vae.fit(x_train, epochs=30, batch_size=128, validation_data=(x_test, None))

PyTorch Implementation

The notebook lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb mirrors the TensorFlow functionality using PyTorch modules.

Simple Autoencoder Architecture

This implementation uses nn.Conv2d and nn.ConvTranspose2d layers:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms

# Data loader

transform = transforms.ToTensor()
train_loader = torch.utils.data.DataLoader(
    datasets.MNIST(".", train=True, download=True, transform=transform),
    batch_size=128, shuffle=True)

# Autoencoder definition

class AutoEncoder(nn.Module):
    def __init__(self):
        super().__init__()
        # Encoder

        self.enc1 = nn.Conv2d(1, 32, 3, stride=2, padding=1)   # 14×14×32

        self.enc2 = nn.Conv2d(32, 64, 3, stride=2, padding=1)  # 7×7×64

        self.fc1  = nn.Linear(7 * 7 * 64, 16)                  # latent dim = 16

        # Decoder

        self.fc2  = nn.Linear(16, 7 * 7 * 64)
        self.dec1 = nn.ConvTranspose2d(64, 32, 3, stride=2, padding=1, output_padding=1) # 14×14×32

        self.dec2 = nn.ConvTranspose2d(32, 1, 3, stride=2, padding=1, output_padding=1)  # 28×28×1

    def forward(self, x):
        # Encode

        x = F.relu(self.enc1(x))
        x = F.relu(self.enc2(x))
        x = x.view(x.size(0), -1)
        z = self.fc1(x)
        # Decode

        x = F.relu(self.fc2(z))
        x = x.view(-1, 64, 7, 7)
        x = F.relu(self.dec1(x))
        x = torch.sigmoid(self.dec2(x))
        return x

model = AutoEncoder()
criterion = nn.BCELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

# Training loop

for epoch in range(20):
    for img, _ in train_loader:
        recon = model(img)
        loss = criterion(recon, img)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    print(f"Epoch {epoch+1}, loss={loss.item():.4f}")

VAE with Reparameterization

The PyTorch VAE explicitly defines encode, reparameterize, and decode methods:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms

# Data loader (same as above)

transform = transforms.ToTensor()
train_loader = torch.utils.data.DataLoader(
    datasets.MNIST(".", train=True, download=True, transform=transform),
    batch_size=128, shuffle=True)

class VAE(nn.Module):
    def __init__(self, latent_dim=2):
        super().__init__()
        # Encoder

        self.enc1 = nn.Conv2d(1, 32, 3, stride=2, padding=1)   # 14×14×32

        self.enc2 = nn.Conv2d(32, 64, 3, stride=2, padding=1)  # 7×7×64

        self.fc_mu   = nn.Linear(7 * 7 * 64, latent_dim)
        self.fc_logv = nn.Linear(7 * 7 * 64, latent_dim)
        # Decoder

        self.fc_dec = nn.Linear(latent_dim, 7 * 7 * 64)
        self.dec1   = nn.ConvTranspose2d(64, 32, 3, stride=2, padding=1, output_padding=1)
        self.dec2   = nn.ConvTranspose2d(32, 1, 3, stride=2, padding=1, output_padding=1)

    def encode(self, x):
        x = F.relu(self.enc1(x))
        x = F.relu(self.enc2(x))
        x = x.view(x.size(0), -1)
        return self.fc_mu(x), self.fc_logv(x)

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        return mu + eps * std

    def decode(self, z):
        x = F.relu(self.fc_dec(z))
        x = x.view(-1, 64, 7, 7)
        x = F.relu(self.dec1(x))
        return torch.sigmoid(self.dec2(x))

    def forward(self, x):
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        recon = self.decode(z)
        return recon, mu, logvar

model = VAE()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

def loss_function(recon, x, mu, logvar):
    BCE = F.binary_cross_entropy(recon, x, reduction='sum')
    KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return BCE + KLD

# Training loop

for epoch in range(20):
    total_loss = 0
    for img, _ in train_loader:
        recon, mu, logvar = model(img)
        loss = loss_function(recon, img, mu, logvar)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f"Epoch {epoch+1}, loss={total_loss:.2f}")

Training Workflow for Image Compression

Both notebooks in lessons/4-ComputerVision/09-Autoencoders/ follow a consistent workflow to implement autoencoders and VAEs for image compression:

1. Load and preprocess data (MNIST or CIFAR-10, normalized to [0, 1])
2. Define the encoder using convolutional or dense layers to reduce spatial dimensions
3. Define the decoder using transposed convolutions to up-sample back to original dimensions
4. Assemble the model by connecting encoder output to decoder input
5. Compile with reconstruction loss (binary_crossentropy or mse)
6. Train the model using the input data as both features and targets
7. Compress images by extracting the encoder's output latent vectors
8. Reconstruct images by feeding latent vectors through the decoder

For VAEs, steps 4-6 include the additional sampling layer and KL divergence loss calculation.

Summary

• Autoencoders use encoder-decoder architectures to compress images into low-dimensional latent spaces and reconstruct them, making them ideal for lossy image compression.
• Variational Autoencoders extend this by learning probabilistic distributions over latents, enabling both compression and generative capabilities through the reparameterization trick.
• The microsoft/AI-For-Beginners repository provides complete implementations in both TensorFlow/Keras (AutoencodersTF.ipynb) and PyTorch (AutoEncodersPyTorch.ipynb).
• Key implementation details include using Conv2D/Conv2DTranspose layers for spatial compression, the reparameterization trick z = μ + ε * exp(logσ² / 2) for VAEs, and combined reconstruction plus KL divergence losses.
• Both frameworks support extracting compressed representations via the encoder and reconstructing images via the decoder for practical compression workflows.

Frequently Asked Questions

What is the difference between an autoencoder and a VAE?

A standard autoencoder learns a deterministic mapping from input to latent code, while a Variational Autoencoder learns a probability distribution (typically Gaussian) parameterized by mean and variance. The VAE uses the reparameterization trick to sample from this distribution, allowing it to generate new images and create smoother, more continuous latent spaces compared to the discrete representations of standard autoencoders.

How do you compress images using a trained autoencoder?

To compress images, pass them through the encoder portion of the trained model to obtain the low-dimensional latent vectors. In TensorFlow, use latent = encoder.predict(x); in PyTorch, use latent = model.encode(x) or the equivalent forward pass through encoder layers. These vectors represent the compressed form and can be stored or transmitted instead of the original pixel data.

Why use binary crossentropy loss for image compression?

Binary crossentropy is used when pixel values are normalized to the range [0, 1], treating each pixel as an independent Bernoulli trial. This works well for grayscale images like MNIST where pixel intensities represent probabilities. For continuous-valued images or RGB data, mean squared error (MSE) is often preferred as it penalizes large pixel deviations more aggressively.

Can autoencoders be used for lossless compression?

Standard autoencoders perform lossy compression because the bottleneck layer forces the network to discard information, and perfect reconstruction is rarely achieved. While the latent representation is compact, the decoder cannot perfectly recover the original input. True lossless compression requires entropy coding techniques that preserve all information, which neural autoencoders do not guarantee.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}