# Building Recurrent Neural Networks (RNNs) for Text Generation: A Complete Guide with PyTorch and TensorFlow

> Master recurrent neural networks (RNNs) for text generation. This guide uses PyTorch and TensorFlow to teach you gated RNNs, embedding layers, and text sampling techniques.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: tutorial
- Published: 2026-08-10

---

**Use gated RNNs (LSTM or GRU) with an embedding layer, stacked recurrent layers, and a linear output projection; train with next-token prediction, then generate text via temperature-scaled softmax sampling or beam search.**

Recurrent Neural Networks (RNNs) and their gated variants **LSTM** and **GRU** remain foundational architectures for character-level and word-level text generation. This guide walks through the complete implementation used in the Microsoft **AI-For-Beginners** curriculum, from data preparation to stochastic text generation with temperature control.

## Understanding the RNN Text Generation Pipeline

The `microsoft/AI-For-Beginners` repository provides a structured pathway through `lessons/5-NLP/16-RNN/` and `lessons/5-NLP/17-GenerativeNetworks/`. The core insight is simple: by training an RNN to predict the next token given all previous tokens, the network learns a probability distribution over sequences that enables **generative sampling**.

### Why RNNs for Generation?

RNNs maintain a **hidden state** that evolves as each token is processed. This state acts as a compressed memory of the sequence so far, allowing the model to capture long-range dependencies without a fixed context window—critical for coherent text generation.

## Step-by-Step Implementation

### 1. Data Preparation

The first step in `GenerativePyTorch.ipynb` and `GenerativeTF.ipynb` involves:

- Loading a raw text corpus
- Building a character or token vocabulary (character-level is more common for beginners)
- Slicing the text into overlapping sequences of fixed length `nchars`

Typical sequence lengths range from **100–200 characters**, with vocabulary sizes of 50–100 for simple character-level models.

### 2. Model Architecture

The standard architecture follows **Embedding → RNN → Linear**:

```python
import torch
import torch.nn as nn
import torch.nn.functional as F

class CharRNN(nn.Module):
    def __init__(self, vocab_size, embed_dim=128, hidden_dim=256):
        super().__init__()
        self.embed = nn.Embedding(vocab_size, embed_dim)
        self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, vocab_size)

    def forward(self, x, hidden=None):
        x = self.embed(x)                     # (B, T, embed_dim)

        out, hidden = self.lstm(x, hidden)    # out: (B, T, hidden_dim)

        logits = self.fc(out)                 # (B, T, vocab_size)

        return logits, hidden

```

**Key architectural choices from the curriculum:**

| Component | Typical Value | Purpose |
|-----------|-------------|---------|
| Embedding size | 128–256 | Dense vector representation of tokens |
| Hidden size | 256–512 | Capacity of the recurrent state |
| Layers | 2–3 | Depth for hierarchical feature learning |
| Dropout | 0.2–0.5 | Regularization between stacked layers |

### 3. Training Loop

The training objective is **next-token prediction** using cross-entropy loss:

```python
def train_step(model, optimizer, x, y):
    model.train()
    optimizer.zero_grad()
    logits, _ = model(x)                     # predict all timesteps

    loss = F.cross_entropy(
        logits.view(-1, logits.size(-1)),
        y.view(-1)
    )
    loss.backward()
    optimizer.step()
    return loss.item()

```

Input `x` and target `y` are offset by one position: the model learns to predict `y[i]` given `x[:i+1]`.

### 4. Text Generation with Temperature

The generation loop in `GenerativePyTorch.ipynb` demonstrates **soft sampling** with temperature control:

```python
def generate_text(model, start_seq, stoi, itos,
                  max_len=500, temperature=1.0):
    model.eval()
    idxs = torch.tensor(
        [stoi[c] for c in start_seq],
        dtype=torch.long
    ).unsqueeze(0)
    hidden = None
    generated = list(start_seq)

    for _ in range(max_len):
        logits, hidden = model(idxs, hidden)
        logits = logits[:, -1, :] / temperature    # temperature scaling

        probs = F.softmax(logits, dim=-1)
        
        next_idx = torch.multinomial(probs, num_samples=1)
        next_char = itos[next_idx.item()]
        
        generated.append(next_char)
        idxs = next_idx.unsqueeze(0)               # autoregressive feedback

        
        if next_char == '<eos>':
            break
    return ''.join(generated)

```

**Temperature effects:**
- **T < 1** (e.g., 0.5): Sharper distribution, more deterministic, repetitive output
- **T = 1**: True model distribution
- **T > 1** (e.g., 1.2–1.5): Flatter distribution, more creative but potentially less coherent

## Advanced Generation: Beam Search

For higher-quality generation, `GPT-PyTorch.ipynb` in `lessons/5-NLP/20-LangModels/` demonstrates **beam search**. Instead of greedy or stochastic sampling, beam search maintains the top-`k` most promising partial sequences at each step, selecting the highest-scoring complete sequence at the end.

Beam widths of **3–5** typically balance quality against computational cost.

## Complete File Reference

| Path | Description |
|------|-------------|
| `lessons/5-NLP/16-RNN/RNNTF.ipynb` | RNN fundamentals and theory (TensorFlow) |
| `lessons/5-NLP/16-RNN/RNNPyTorch.ipynb` | RNN fundamentals and theory (PyTorch) |
| `lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.ipynb` | Full TensorFlow text generation pipeline |
| `lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb` | Complete PyTorch implementation with temperature sampling |
| `lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb` | Beam search and advanced generation techniques |

## Summary

- **Building RNNs for text generation** requires three components: an embedding layer, gated recurrent layers (LSTM/GRU), and a linear output projection
- Training uses next-token prediction with cross-entropy loss on overlapping sequences
- Generation employs **temperature scaling** to control the randomness/creativity trade-off
- **Beam search** from `GPT-PyTorch.ipynb` provides an alternative for higher-quality deterministic output
- The `microsoft/AI-For-Beginners` repository provides complete, runnable notebooks in both PyTorch and TensorFlow

## Frequently Asked Questions

### What is the difference between LSTM and GRU for text generation?

**LSTM (Long Short-Term Memory)** uses separate input, forget, and output gates with a cell state, providing more flexible memory control. **GRU (Gated Recurrent Unit)** merges the cell and hidden states with only reset and update gates, making it computationally cheaper with fewer parameters. Both are implemented in `RNNPyTorch.ipynb` and `RNNTF.ipynb`; GRU trains faster while LSTM sometimes captures longer dependencies better.

### How does temperature affect text generation quality?

Temperature scales logits before softmax: `logits / T`. Lower temperature (T < 1) concentrates probability mass on the most likely tokens, producing repetitive but coherent text. Higher temperature (T > 1) flattens the distribution, increasing diversity but risking grammatical errors or nonsense. The curriculum recommends **0.5–1.5** as a practical range.

### Can I use word-level instead of character-level tokens?

Yes. Replace character vocabulary with a word tokenizer (or subword tokenizer like BPE). The architecture remains identical—only `vocab_size` increases significantly. Character-level models have smaller vocabularies (~100) but longer sequences; word-level models have larger vocabularies (10K–100K) but shorter sequences. The `GenerativePyTorch.ipynb` notebook uses characters for simplicity.

### When should I use beam search instead of temperature sampling?

Use **beam search** when you want the most likely sequence under the model (e.g., for completion tasks where correctness matters). Use **temperature sampling** when you want creative, diverse outputs (e.g., story generation, brainstorming). Beam search is deterministic; temperature sampling is stochastic. Both techniques are covered in the AI-For-Beginners curriculum.