# How to Implement Recurrent Neural Networks for Text Generation with RNNPyTorch.ipynb

> Learn to implement recurrent neural networks for text generation using PyTorch. This guide adapts sequence classification models for autoregressive text generation with simple modifications.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: tutorial
- Published: 2026-08-08

---

**The RNNPyTorch.ipynb notebook in Microsoft's AI-For-Beginners repository provides a complete PyTorch implementation of recurrent neural networks that can be adapted from sequence classification to autoregressive text generation by replacing the output head and modifying the training objective.**

This tutorial walks through the end-to-end workflow found in `lessons/5-NLP/16-RNN/RNNPyTorch.ipynb` from the [microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners) curriculum. You will learn how to leverage the notebook's `LSTMClassifier` architecture, `train_epoch` loop, and packed sequence utilities to build models that generate text token-by-token.

## Understanding the RNN Architecture in RNNPyTorch.ipynb

The notebook demonstrates three variants of recurrent networks for processing sequential text data. Each implementation follows a consistent pattern: embedding layer → recurrent cell → aggregation → classification head.

### Data Loading and Vocabulary Construction

The notebook utilizes the `torchnlp` utility module ([`torchnlp.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/torchnlp.py)) to load a news-category dataset and construct a token-to-index vocabulary. The `padify` function handles variable-length sequences by padding them to uniform tensors suitable for batching.

```python

# Conceptual flow from the notebook's data preparation

vocab = build_vocab(texts)  # Maps tokens to indices

padded_batch = padify(raw_texts, vocab)  # Returns (batch, seq_len) tensor

```

### Embedding and Recurrent Layers

Every model begins with `torch.nn.Embedding` to convert token indices into dense vectors, followed by a recurrent cell. The notebook compares three PyTorch implementations:

- **Simple RNN**: `torch.nn.RNN` processes sequences step-by-step but suffers from vanishing gradients on long texts.
- **LSTM**: `torch.nn.LSTM` introduces forget, input, and output gates to stabilize gradient flow across longer sequences.
- **GRU**: A gated alternative mentioned in the notebook's markdown cells.

All cells share weights across time steps, propagating a hidden state `Sᵢ` that captures temporal dependencies.

### Classification Head vs. Generation Head

The notebook's default implementations (`RNNClassifier` and `LSTMClassifier`) aggregate hidden states via mean pooling or final-state extraction, then project to class logits using `torch.nn.Linear`. For text generation, you replace this classification head with a vocabulary projection layer that predicts the next token probability distribution.

## Building Text Generation Models from the Classification Base

While `RNNPyTorch.ipynb` focuses on **sequence classification**, the same architectural components support **autoregressive text generation** with minimal modifications.

### Converting the Classifier to a Generator

Replace the class-projection layer with a vocabulary projection and adjust the forward pass to return per-token logits:

```python
import torch
import torch.nn as nn

class LSTMGenerator(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.rnn = nn.LSTM(embed_dim, hidden_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, vocab_size)  # next-token logits

    def forward(self, x, lengths=None):
        x = self.embedding(x)
        if lengths is not None:
            x = nn.utils.rnn.pack_padded_sequence(
                x, lengths, 
                batch_first=True, 
                enforce_sorted=False
            )
            x, (h, c) = self.rnn(x)
            x, _ = nn.utils.rnn.pad_packed_sequence(x, batch_first=True)
        else:
            x, (h, c) = self.rnn(x)
        return self.fc(x)  # shape: (batch, seq_len, vocab_size)

```

### Autoregressive Inference Loop

During generation, feed the model's output back as the next input, sampling from the probability distribution until an end-of-sequence token or length limit is reached:

```python
def generate(model, start_token, vocab, max_len=50, temperature=1.0):
    model.eval()
    tokens = [start_token]
    hidden = None
    
    for _ in range(max_len):
        input_tensor = torch.tensor([tokens[-1]]).unsqueeze(0)  # (1, 1)

        logits = model(input_tensor)
        logits = logits[:, -1, :] / temperature
        probs = torch.softmax(logits, dim=-1)
        next_token = torch.multinomial(probs, 1).item()
        tokens.append(next_token)
        
        if next_token == vocab['<eos>']:
            break
            
    return tokens

```

Training this generator requires shifting the target sequence by one position (predicting `x[:, 1:]` from `x[:, :-1]`) and using cross-entropy loss, following the same `train_epoch` pattern defined in the notebook.

## Core Implementation Patterns from the Notebook

The following excerpts from `lessons/5-NLP/16-RNN/RNNPyTorch.ipynb` demonstrate the fundamental building blocks used in both classification and generation tasks.

### Simple RNN Classifier

This implementation averages hidden states across the time dimension for classification:

```python
class RNNClassifier(torch.nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):
        super().__init__()
        self.embedding = torch.nn.Embedding(vocab_size, embed_dim)
        self.rnn = torch.nn.RNN(embed_dim, hidden_dim, batch_first=True)
        self.fc = torch.nn.Linear(hidden_dim, num_class)

    def forward(self, x):
        # x shape: (batch, seq_len)

        x = self.embedding(x)               # (batch, seq_len, embed_dim)

        x, _ = self.rnn(x)                   # (batch, seq_len, hidden_dim)

        return self.fc(x.mean(dim=1))        # average pooling over time

```

*Source:* Lines 63-71 in `RNNPyTorch.ipynb`

### LSTM Classifier for Longer Sequences

Using the final hidden state rather than mean pooling often yields better performance for LSTM-based models:

```python
class LSTMClassifier(torch.nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):
        super().__init__()
        self.embedding = torch.nn.Embedding(vocab_size, embed_dim)
        self.rnn = torch.nn.LSTM(embed_dim, hidden_dim, batch_first=True)
        self.fc = torch.nn.Linear(hidden_dim, num_class)

    def forward(self, x):
        x = self.embedding(x)
        x, (h, _) = self.rnn(x)          # h shape: (1, batch, hidden_dim)

        return self.fc(h.squeeze(0))      # use final hidden state

```

*Source:* Lines 75-84 in `RNNPyTorch.ipynb`

### Packed Sequence Optimization

For variable-length batches, the notebook implements `LSTMPackClassifier` using `torch.nn.utils.rnn.pack_padded_sequence` to skip padding tokens in the computation graph:

```python
class LSTMPackClassifier(torch.nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):
        super().__init__()
        self.embedding = torch.nn.Embedding(vocab_size, embed_dim)
        self.rnn = torch.nn.LSTM(embed_dim, hidden_dim,
                                 batch_first=True)
        self.fc = torch.nn.Linear(hidden_dim, num_class)

    def forward(self, x, lengths):
        x = self.embedding(x)
        packed = torch.nn.utils.rnn.pack_padded_sequence(
                    x, lengths, batch_first=True, enforce_sorted=False)
        packed, (h, _) = self.rnn(packed)
        return self.fc(h.squeeze(0))

```

*Source:* Lines 30-37 in `RNNPyTorch.ipynb`

### Training Loop Implementation

The `train_epoch` function handles both standard and packed-sequence models:

```python
def train_epoch(net, loader, lr=0.001):
    optimizer = torch.optim.Adam(net.parameters(), lr=lr)
    criterion = torch.nn.CrossEntropyLoss()
    net.train()
    for step, (labels, batch, *rest) in enumerate(loader):
        optimizer.zero_grad()
        # For packed seq models, `rest` contains lengths

        if rest:
            outputs = net(batch.to(device), rest[0].to('cpu'))
        else:
            outputs = net(batch.to(device))
        loss = criterion(outputs, labels.to(device))
        loss.backward()
        optimizer.step()
        if step % 100 == 0:
            acc = (outputs.argmax(dim=1) == labels.to(device)).float().mean()
            print(f"{step*len(batch)}: acc={acc.item():.4f}")

```

*Source:* Training cell L42-L45 in `RNNPyTorch.ipynb`

## Summary

- **RNNPyTorch.ipynb** provides complete implementations of `RNNClassifier`, `LSTMClassifier`, and `LSTMPackClassifier` using PyTorch's `nn.RNN` and `nn.LSTM` modules.
- The notebook's architecture separates embedding layers, recurrent cells, and output heads, making it straightforward to adapt classification models for text generation.
- **Packed sequences** via `torch.nn.utils.rnn.pack_padded_sequence` eliminate unnecessary computation on padding tokens, significantly improving training efficiency for variable-length text.
- To convert the provided classifiers into generators, replace the classification `nn.Linear` layer with a vocabulary-sized output layer and implement autoregressive sampling using `torch.multinomial`.
- The `train_epoch` function and data loading utilities in [`torchnlp.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/torchnlp.py) support both classification and generation training loops with minimal modification.

## Frequently Asked Questions

### What is the difference between RNN and LSTM in the notebook?

The `RNNClassifier` uses `torch.nn.RNN`, which implements a simple recurrent cell that struggles with long-term dependencies due to vanishing gradients. The `LSTMClassifier` uses `torch.nn.LSTM`, which adds forget, input, and output gates to control information flow, allowing the model to retain context over longer sequences. According to the notebook's markdown cells, LSTM is preferred for text tasks exceeding 10-20 tokens.

### How does packed padding improve RNN performance?

The `LSTMPackClassifier` implementation uses `torch.nn.utils.rnn.pack_padded_sequence` to remove padding tokens from the computation graph before feeding data to the LSTM. This prevents the recurrent cell from processing meaningless zero-vectors, reducing both computation time and memory usage during training. The notebook demonstrates this pattern in lines 30-37.

### Can I use the same model for both classification and generation?

Yes. The architectural backbone—embedding layer plus LSTM—is identical for both tasks. To switch from classification to generation, replace the final `nn.Linear(hidden_dim, num_class)` with `nn.Linear(hidden_dim, vocab_size)` and modify the forward pass to return sequences rather than aggregated vectors. The training objective changes from cross-entropy over classes to next-token prediction.

### What dataset does the notebook use for training?

The notebook utilizes a news-category dataset loaded through the [`torchnlp.py`](https://github.com/microsoft/AI-For-Beginners/blob/main/torchnlp.py) utility module, which provides `load_dataset` and `padify` functions. This dataset contains news headlines labeled by category (such as sports, politics, and entertainment), making it suitable for training sequence classification models that can later be repurposed for text generation tasks.