# How to Use RNNs for Text Generation in the AI-For-Beginners NLP Curriculum

> Learn to use RNNs for text generation in NLP with the AI-For-Beginners curriculum. Master sequence prediction using LSTMs, GRUs, and sampling.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: tutorial
- Published: 2026-08-06

---

**RNNs generate text by maintaining a hidden state that encodes sequence history and predicting the next token through an embedding layer, recurrent cell (LSTM/GRU), and softmax output, with sampling strategies like temperature-controlled multinomial selection to control creativity.**

The Microsoft AI-For-Beginners repository provides a complete pedagogical path for understanding how to use RNNs for text generation within its 5-NLP module. By progressing from basic recurrent architectures in lesson 16 to full generative implementations in lesson 17, the curriculum demonstrates how these networks learn character-level patterns to produce coherent sequences. This guide extracts the essential architectural patterns and implementation details directly from the source notebooks.

## Architectural Foundations of RNN Text Generation

### Sequential Processing and Hidden States

Unlike feedforward networks, RNNs process sequences step-by-step, using a **hidden state** that acts as memory of previous inputs. In `lessons/5-NLP/16-RNN/RNNTF.ipynb` and `lessons/5-NLP/16-RNN/RNNPyTorch.ipynb`, the curriculum establishes that each time step receives both the current input token and the previous hidden state, producing an updated state that encodes the entire sequence history. This mechanism is essential for generative tasks because the probability of the next character depends on all preceding characters.

### The Three-Stage Generative Pipeline

The notebooks in `lessons/5-NLP/17-GenerativeNetworks/` implement a consistent three-stage architecture for text generation:

1. **Embedding Layer** – Maps integer-encoded characters to dense vectors of fixed dimension (`embed_dim`).
2. **Recurrent Layer** – An LSTM or GRU processes the embedded sequence with `return_sequences=True` to output hidden states at every time step.
3. **Output Projection** – A fully connected layer maps each hidden state to logits over the vocabulary, followed by a softmax to produce a probability distribution for next-token prediction.

## PyTorch Implementation of Character-Level Generation

The [GenerativePyTorch.ipynb](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb) notebook provides a complete implementation using an LSTM-based character-level model. The architecture is encapsulated in the `CharRNN` class, which combines embedding, recurrence, and projection layers.

```python
import torch
import torch.nn as nn
import numpy as np

class CharRNN(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim):
        super().__init__()
        self.embed = nn.Embedding(vocab_size, embed_dim)
        self.rnn = nn.LSTM(embed_dim, hidden_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, vocab_size)
    
    def forward(self, x, hidden=None):
        x = self.embed(x)  # (batch, seq_len, embed_dim)

        out, hidden = self.rnn(x, hidden)  # out: (batch, seq_len, hidden_dim)

        logits = self.fc(out)  # (batch, seq_len, vocab_size)

        return logits, hidden

```

Training involves minimizing the **cross-entropy loss** between the predicted logits and the target sequence shifted by one position. The optimization loop uses `torch.optim.Adam` to update the embedding, LSTM, and output layer parameters.

## TensorFlow/Keras Implementation

For TensorFlow users, [GenerativeTF.ipynb](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.ipynb) demonstrates an equivalent implementation using the Keras Sequential API. The model stacks an `Embedding` layer, an `LSTM` layer with `return_sequences=True`, and a `Dense` output layer.

```python
import tensorflow as tf
import numpy as np

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embed_dim),
    tf.keras.layers.LSTM(hidden_dim, return_sequences=True),
    tf.keras.layers.Dense(vocab_size)
])

model.compile(
    optimizer='adam',
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
)

```

The training process uses `model.fit()` with sparse categorical crossentropy, directly optimizing the network to predict the next character index given the current sequence.

## Generation and Sampling Strategies

Once trained, both frameworks implement a `generate()` function that iteratively predicts the next token and appends it to the output sequence. The key challenge is converting the softmax distribution into a discrete choice without collapsing into repetitive loops.

### Temperature-Controlled Sampling

The curriculum introduces **temperature scaling** to control the randomness of generation. By dividing the logits by a `temperature` parameter before applying softmax, values below 1.0 sharpen the distribution (more conservative output), while values above 1.0 flatten it (more random output).

```python
def generate(start_seq, length=200, temperature=1.0):
    model.eval()
    input_seq = torch.tensor([char2idx[c] for c in start_seq], 
                             dtype=torch.long).unsqueeze(0)
    hidden = None
    generated = start_seq
    
    for _ in range(length):
        logits, hidden = model(input_seq, hidden)
        logits = logits[:, -1, :] / temperature  # Scale by temperature

        probs = torch.softmax(logits, dim=-1)
        next_idx = torch.multinomial(probs, num_samples=1).item()
        next_char = idx2char[next_idx]
        generated += next_char
        input_seq = torch.tensor([[next_idx]], dtype=torch.long)
    
    return generated

```

### Multinomial vs. Greedy Selection

While **greedy sampling** always selects the highest-probability token, the notebooks emphasize **multinomial sampling** using `torch.multinomial` or `np.random.choice` with probability weights. This stochastic approach prevents the model from getting stuck in repetitive loops and produces more natural, varied text.

## Evolution to Modern Architectures

For comparison, [GPT-PyTorch.ipynb](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb) implements transformer-based generation using attention mechanisms instead of recurrence. While RNNs process sequences sequentially (limiting parallelization and long-range dependencies), this file demonstrates how modern architectures scale the same next-token prediction objective to much larger contexts, providing students with a clear evolutionary path from RNNs to GPT-style models.

## Summary

- **RNNs capture sequence order** through hidden states that propagate across time steps, making them suitable for text generation tasks where context matters.
- **The generative architecture** consists of an embedding layer, an LSTM/GRU recurrent layer with `return_sequences=True`, and a dense output layer that produces vocabulary logits at each position.
- **Training** uses next-token prediction with cross-entropy loss, optimizing the network to minimize the negative log-likelihood of the target sequence.
- **Temperature scaling** and **multinomial sampling** are essential techniques for controlling the creativity and diversity of generated text.
- The Microsoft AI-For-Beginners curriculum provides complete implementations in both [PyTorch](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb) and [TensorFlow](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.ipynb), progressing from basic RNN concepts in [lesson 16](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/16-RNN/RNNTF.ipynb) to full generative systems.

## Frequently Asked Questions

### What is the difference between using an LSTM and a GRU for text generation in these lessons?

Both LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) are implemented as alternatives to vanilla RNNs to solve the vanishing gradient problem. According to the source code in `GenerativePyTorch.ipynb`, LSTMs use separate cell states and gates (input, forget, output) to better preserve long-range dependencies, while GRUs merge these gates for computational efficiency. For character-level generation with relatively short sequences, both perform similarly, but LSTMs are generally preferred for capturing subtle long-term patterns.

### Why does the code use character-level instead of word-level generation?

The notebooks in `lessons/5-NLP/17-GenerativeNetworks/` use character-level modeling to keep the vocabulary size small and manageable for educational purposes. As noted in the implementation, character-level models learn spelling, punctuation, and word morphology without requiring large embedding matrices, making them feasible to train on CPU in a classroom setting while still demonstrating the core principles of RNN text generation.

### How does temperature affect the quality of generated text?

Temperature scales the logits before applying the softmax function; a lower temperature (e.g., 0.5) makes the probability distribution peakier, causing the model to select high-probability tokens more consistently and producing conservative, repetitive text. A higher temperature (e.g., 1.5) flattens the distribution, increasing the chance of selecting lower-probability tokens and resulting in more diverse but potentially incoherent output. The lessons recommend tuning this parameter between 0.7 and 1.0 for balanced creativity.

### Can these RNN models be used for tasks other than text generation?

Yes, the same RNN architecture demonstrated in `lessons/5-NLP/16-RNN/` supports various sequence modeling tasks including sentiment analysis, named entity recognition, and machine translation. The curriculum notes that while the generative notebooks focus on next-character prediction, the underlying `CharRNN` class and training loop can be adapted for sequence classification by using only the final hidden state and replacing the output layer with a classification head.