# How to Use Autoencoders and VAEs for Dimensionality Reduction: A Practical Guide > Learn how to use autoencoders and VAEs for dimensionality reduction. Compress data into low-dimensional vectors for visualization, clustering, and machine learning with this practical guide. - Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners) - Tags: how-to-guide - Published: 2026-08-06 --- **Autoencoders and Variational Autoencoders (VAEs) compress high-dimensional data into low-dimensional latent vectors by training neural networks to reconstruct their inputs through a bottleneck layer, producing embeddings suitable for visualization, clustering, and downstream machine learning tasks.** The **microsoft/AI-For-Beginners** repository provides complete implementations demonstrating how to leverage these architectures for effective dimensionality reduction. By forcing data through a compact latent space, these models learn to preserve salient features while discarding noise, outperforming traditional linear methods like PCA on complex datasets. ## Understanding the Autoencoder Architecture An **autoencoder** is a neural network trained to copy its input to its output. The critical constraint is the **bottleneck**—a latent layer with fewer neurons than the input dimension—which forces the network to learn a compressed representation. ### The Encoder-Bottleneck-Decoder Pipeline According to the source code in [`lessons/4-ComputerVision/09-Autoencoders/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/09-Autoencoders/README.md), the architecture consists of three components: 1. **Encoder** – A stack of convolutional or fully-connected layers that compresses the input into a latent vector **z** of size *d* < *input_dim*. 2. **Bottleneck** – The latent vector containing the compressed representation. Its dimensionality directly determines the degree of reduction. 3. **Decoder** – Mirrors the encoder using transposed convolutions or dense layers to reconstruct the original input from **z**. The network minimizes a **reconstruction loss** (typically Mean Squared Error) measuring pixel-wise or feature-wise differences between input and output. In `lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb`, this architecture is implemented using TensorFlow/Keras convolutional layers for MNIST digit compression. ### Why Autoencoders Surpass PCA for Dimensionality Reduction **Autoencoders reduce dimensionality more effectively than Principal Component Analysis** because they capture non-linear relationships and spatial hierarchies: - **Non-linear encoding** – Activation functions (ReLU, Sigmoid) enable the encoder to map data onto complex manifolds that linear projections cannot represent. - **Spatial awareness** – Convolutional layers preserve locality and hierarchical features, whereas PCA treats each pixel or feature independently. ## How Variational Autoencoders (VAEs) Enhance Latent Representations A **Variational Autoencoder** extends the classic architecture by treating the latent space as a **probabilistic distribution** rather than a deterministic point. This approach, detailed in `lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb`, generates smoother, more interpretable embeddings. ### Probabilistic Latent Spaces and the Reparameterization Trick Unlike standard autoencoders that output a fixed vector, the VAE encoder predicts two parameters for each latent dimension: - **Mean (μ)** – The center of the Gaussian distribution. - **Log-variance (log σ²)** – The uncertainty or spread. To enable backpropagation through the random sampling process, VAEs employ the **reparameterization trick**: ```python z = μ + σ * ε ``` Here, **ε** is sampled from a standard normal distribution 𝒩(0, I), while μ and σ are learned parameters. This formulation allows gradients to flow through the stochastic node during training. ### The VAE Loss Function: Reconstruction Plus KL Divergence The VAE optimizes a composite loss function combining two objectives: 1. **Reconstruction loss** – Identical to classic autoencoders (e.g., MSE or Binary Cross-Entropy), ensuring the decoded output matches the input. 2. **KL Divergence** – A regularization term that forces the learned distribution to stay close to a unit Gaussian: ``` KL(N(μ, σ) || N(0, 1)) = -0.5 * Σ(1 + log(σ²) - μ² - σ²) ``` This regularization encourages a smooth, continuous latent space where similar inputs map to nearby regions, making the embeddings more robust for downstream dimensionality reduction tasks. ## Practical Implementation: Extracting Low-Dimensional Embeddings To use autoencoders and VAEs for dimensionality reduction, you must train the model, extract the latent vectors, and apply them to downstream tasks. ### TensorFlow/Keras Convolutional Autoencoder The following implementation from `lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb` demonstrates building a convolutional autoencoder for image compression: ```python import tensorflow as tf from tensorflow.keras import layers, models def build_encoder(latent_dim=32): inputs = layers.Input(shape=(28, 28, 1)) x = layers.Conv2D(32, 3, activation='relu', padding='same')(inputs) x = layers.MaxPooling2D(2, padding='same')(x) x = layers.Conv2D(64, 3, activation='relu', padding='same')(x) x = layers.MaxPooling2D(2, padding='same')(x) x = layers.Flatten()(x) latent = layers.Dense(latent_dim, name='latent')(x) return models.Model(inputs, latent, name='encoder') def build_decoder(latent_dim=32, shape_before_flatten=(7, 7, 64)): latent_inputs = layers.Input(shape=(latent_dim,)) x = layers.Dense(7 * 7 * 64)(latent_inputs) x = layers.Reshape(shape_before_flatten)(x) x = layers.Conv2DTranspose(64, 3, strides=2, activation='relu', padding='same')(x) x = layers.Conv2DTranspose(32, 3, strides=2, activation='relu', padding='same')(x) outputs = layers.Conv2D(1, 3, activation='sigmoid', padding='same')(x) return models.Model(latent_inputs, outputs, name='decoder') # Build and compile encoder = build_encoder() decoder = build_decoder() autoencoder = models.Model(encoder.input, decoder(encoder.output)) autoencoder.compile(optimizer='adam', loss='mse') # Train autoencoder.fit(x_train, x_train, epochs=30, batch_size=256) # Extract reduced-dimension features z = encoder.predict(x_test) # Shape: (n_samples, 32) ``` ### PyTorch Variational Autoencoder For probabilistic embeddings, the PyTorch implementation in `lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb` defines a fully-connected VAE: ```python import torch import torch.nn as nn import torch.nn.functional as F class VAE(nn.Module): def __init__(self, latent_dim=20): super().__init__() self.fc1 = nn.Linear(28*28, 400) self.fc_mu = nn.Linear(400, latent_dim) self.fc_logvar = nn.Linear(400, latent_dim) self.fc2 = nn.Linear(latent_dim, 400) self.fc3 = nn.Linear(400, 28*28) def encode(self, x): h = F.relu(self.fc1(x)) return self.fc_mu(h), self.fc_logvar(h) def reparameterize(self, mu, logvar): std = torch.exp(0.5 * logvar) eps = torch.randn_like(std) return mu + eps * std def decode(self, z): h = F.relu(self.fc2(z)) return torch.sigmoid(self.fc3(h)) def forward(self, x): mu, logvar = self.encode(x.view(-1, 784)) z = self.reparameterize(mu, logvar) return self.decode(z), mu, logvar def loss_function(recon_x, x, mu, logvar): BCE = F.binary_cross_entropy(recon_x, x.view(-1, 784), reduction='sum') KLD = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp()) return BCE + KLD # Training loop model = VAE() optimizer = torch.optim.Adam(model.parameters(), lr=1e-3) for epoch in range(10): for batch, _ in train_loader: optimizer.zero_grad() recon, mu, logvar = model(batch) loss = loss_function(recon, batch, mu, logvar) loss.backward() optimizer.step() # Extract deterministic embeddings for dimensionality reduction with torch.no_grad(): mu, _ = model.encode(x_test.view(-1, 784)) latent = mu.numpy() # Use mean as reduced-dimension features ``` ### Downstream Dimensionality Reduction Pipeline Once you have trained your encoder, extract the latent vectors and apply them to visualization or clustering tasks: ```python # 1. Encode the dataset z_train = encoder.predict(x_train) # TensorFlow # OR z_train = model.encode(x_train.view(-1, 784))[0].numpy() # PyTorch # 2. Visualize with UMAP or t-SNE import umap import matplotlib.pyplot as plt reducer = umap.UMAP(n_components=2, random_state=42) z_2d = reducer.fit_transform(z_train) plt.scatter(z_2d[:, 0], z_2d[:, 1], c=y_train, cmap='Spectral', s=5) plt.title('2D Embedding from Autoencoder Latent Space') plt.show() ``` ## Summary - **Autoencoders** learn compressed representations through an encoder-bottleneck-decoder architecture, using reconstruction loss to train the network to preserve essential features in the latent space. - **VAEs** extend this approach by modeling the latent space as a probability distribution, using the reparameterization trick and KL divergence to create smooth, continuous embeddings. - **Dimensionality reduction** is achieved by extracting the latent vectors (deterministic **z** for autoencoders, **μ** for VAEs) and using them as low-dimensional features for visualization, clustering, or downstream classification. - The **microsoft/AI-For-Beginners** repository provides complete TensorFlow and PyTorch implementations in `lessons/4-ComputerVision/09-Autoencoders/` demonstrating these techniques on standard datasets like MNIST. ## Frequently Asked Questions ### What is the difference between an autoencoder and a VAE for dimensionality reduction? A classic autoencoder produces a deterministic latent vector **z**, while a VAE outputs parameters (mean **μ** and variance **σ**) of a probability distribution. For dimensionality reduction, autoencoders provide fixed embeddings, whereas VAEs offer probabilistic embeddings that can be sampled. In practice, use the mean vector **μ** from a VAE as the deterministic reduced-dimension feature, which tends to produce more robust and smoothly interpolating latent spaces than standard autoencoders. ### How do you choose the size of the latent dimension? The latent dimension represents a trade-off between compression and reconstruction quality. Start with a dimension significantly smaller than your input (e.g., 32 for 784-pixel MNIST images) and monitor the reconstruction loss. If the loss is high, increase the dimension; if you need more aggressive compression, decrease it. According to the implementations in `lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb`, values between 16 and 64 typically work well for image datasets. ### Can autoencoders be used for non-image data? Yes, autoencoders and VAEs are framework-agnostic and work for any high-dimensional data including tabular features, time series, and text embeddings. Replace the convolutional layers in the computer vision examples with fully-connected (dense) layers for tabular data, or use 1D convolutions and LSTM layers for sequential data. The core principle—compressing through a bottleneck and reconstructing—remains identical regardless of input modality. ### Why add KL divergence loss in VAEs if we only want dimensionality reduction? The **KL divergence** term acts as a regularizer that prevents overfitting to specific training examples and encourages the latent space to follow a standard normal distribution. This creates a smooth, continuous space where similar inputs cluster together and interpolation between points produces meaningful outputs. Without this term, the VAE behaves like a standard autoencoder with a discontinuous latent space, potentially creating "holes" where decoder reconstructions become unreliable.