# NLP Techniques in Microsoft AI for Beginners: Complete Guide to Lessons 13-20

> Explore NLP techniques in Microsoft AI for Beginners Lessons 13-20. Learn Bag-of-Words, TF-IDF, Transformers, and GPT with TensorFlow and PyTorch.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: tutorial
- Published: 2026-08-22

---

**The AI for Beginners curriculum teaches eight progressive NLP techniques across Lessons 13-20, ranging from classical Bag-of-Words and TF-IDF text representation to modern Transformer architectures and GPT-based large language models, with dual implementations in TensorFlow and PyTorch.**

The Microsoft AI for Beginners repository provides a comprehensive, hands-on introduction to natural language processing through eight sequential lessons in the `lessons/5-NLP/` directory. Each lesson builds upon the previous, guiding learners from sparse text representations to state-of-the-art large language models using both TensorFlow and PyTorch implementations.

## Text Representation Fundamentals (Lessons 13-14)

The curriculum begins with foundational techniques for converting raw text into numerical representations that machine learning models can process.

### Lesson 13 – Bag-of-Words and TF-IDF

Lesson 13 introduces **Bag-of-Words (BoW)** and **Term Frequency-Inverse Document Frequency (TF-IDF)** to create sparse numeric vectors from text. These classical techniques represent documents based on word occurrence statistics without considering semantic meaning or word order.

The notebooks `lessons/5-NLP/13-TextRep/TextRepresentationTF.ipynb` and `TextRepresentationPyTorch.ipynb` demonstrate vectorization using scikit-learn:

```python
from sklearn.feature_extraction.text import TfidfVectorizer

documents = ["I love AI.", "AI for beginners is fun."]
vectorizer = TfidfVectorizer()
tfidf_matrix = vectorizer.fit_transform(documents)

print(tfidf_matrix.toarray())
print(vectorizer.get_feature_names_out())

```

### Lesson 14 – Dense Word Embeddings

Moving beyond sparse vectors, Lesson 14 teaches **Word Embeddings**—dense, continuous-space representations that capture semantic relationships between words. The lesson implements embedding layers that map vocabulary indices to low-dimensional vectors.

In `lessons/5-NLP/14-Embeddings/EmbeddingsTF.ipynb`, the curriculum demonstrates TensorFlow's `Embedding` layer:

```python
import tensorflow as tf

vocab_size = 5000
embed_dim = 128
embedding = tf.keras.layers.Embedding(vocab_size, embed_dim)

sample_ids = tf.constant([[1, 5, 23, 7]])
embedded = embedding(sample_ids)   # shape: (1, 4, 128)

print(embedded.shape)

```

## Neural Language Modeling (Lessons 15-17)

The intermediate section transitions to neural network architectures for understanding and generating text sequences.

### Lesson 15 – Continuous Bag-of-Words (CBoW)

Lesson 15 implements the **Continuous Bag-of-Words** architecture, a shallow neural network that predicts a target word based on its surrounding context words. This introduces the concept of learning distributed representations through prediction tasks.

The `lessons/5-NLP/15-LanguageModeling/CBoW-PyTorch.ipynb` notebook defines the architecture:

```python
import torch
import torch.nn as nn

class CBoW(nn.Module):
    def __init__(self, vocab_sz, embed_sz):
        super().__init__()
        self.emb = nn.Embedding(vocab_sz, embed_sz)
        self.linear = nn.Linear(embed_sz, vocab_sz)

    def forward(self, context_idxs):
        embeds = self.emb(context_idxs).mean(dim=1)
        out = self.linear(embeds)
        return out

```

### Lesson 16 – Recurrent Neural Networks (RNNs)

Lesson 16 addresses sequential dependencies in text using **Recurrent Neural Networks** and LSTM variants. The curriculum demonstrates next-character prediction to illustrate how RNNs maintain hidden state across time steps.

The implementation in `lessons/5-NLP/16-RNN/RNNTF.ipynb` shows the layer stack:

```python
model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, 256),
    tf.keras.layers.SimpleRNN(1024, return_sequences=True),
    tf.keras.layers.Dense(vocab_size)
])
model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              optimizer='adam')

```

### Lesson 17 – Generative Text Networks

Building on RNN foundations, Lesson 17 focuses on **Generative Language Models** that produce novel text sequences. The lesson trains models to generate coherent sentences by sampling from the learned probability distribution over vocabulary.

## Modern NLP Architectures (Lessons 18-20)

The final section covers contemporary approaches using attention mechanisms and pre-trained models.

### Lesson 18 – Transformer Architectures

Lesson 18 introduces the **Transformer** encoder-decoder architecture with self-attention mechanisms. Rather than building from scratch, the curriculum leverages Hugging Face's `transformers` library to load pre-trained models like BERT and demonstrates fine-tuning techniques.

From `lessons/5-NLP/18-Transformers/TransformersPyTorch.ipynb`:

```python
from transformers import pipeline

generator = pipeline("text-generation", model="gpt2")
print(generator("Artificial intelligence is", max_length=50, num_return_sequences=1))

```

### Lesson 19 – Named Entity Recognition (NER)

Lesson 19 applies sequence labeling to **Named Entity Recognition**, teaching models to identify and classify entities such as persons, organizations, and locations. The implementation uses either Bi-LSTM-CRF architectures or pre-trained Transformer models for token-level classification.

The TensorFlow implementation in `lessons/5-NLP/19-NER/NER-TF.ipynb` shows prediction patterns:

```python
import tensorflow as tf

# assuming `token_ids` and `attention_mask` are pre-computed

model = tf.keras.models.load_model('ner_tf_model')
pred = model.predict([token_ids, attention_mask])
entities = tf.argmax(pred, axis=-1)

```

### Lesson 20 – Large Language Models and GPT

The final lesson explores **Large Language Models (LLMs)** using OpenAI-GPT architectures. The curriculum covers prompt engineering strategies, sampling methods (greedy decoding, stochastic sampling with temperature, top-k, and nucleus/top-p sampling), beam search, and fine-tuning approaches.

In `lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb`, the notebook demonstrates advanced generation parameters:

```python
from transformers import pipeline

generator = pipeline("text-generation", model="openai-gpt")
prompt = "Translate English to French: cat =>"
out = generator(prompt,
                max_length=30,
                num_return_sequences=3,
                do_sample=True,
                temperature=0.8,
                top_k=50,
                top_p=0.95)
print(out)

```

## Summary

The Microsoft AI for Beginners NLP curriculum provides a structured progression through essential natural language processing techniques:

- **Lessons 13-14** establish text representation using sparse TF-IDF vectors and dense word embeddings
- **Lessons 15-17** build neural language models with CBoW, RNNs, and generative architectures
- **Lessons 18-20** advance to Transformer models, Named Entity Recognition, and Large Language Model interaction

Each technique includes working implementations in both TensorFlow and PyTorch, with complete notebooks available in the `lessons/5-NLP/` directory of the repository.

## Frequently Asked Questions

### What is the difference between Bag-of-Words and Word Embeddings in the curriculum?

Lesson 13 teaches **Bag-of-Words** and TF-IDF, which create sparse, high-dimensional vectors based on word frequency counts without semantic context. Lesson 14 introduces **Word Embeddings**, which map words into dense, low-dimensional continuous vectors where semantic relationships are preserved—similar words occupy nearby positions in vector space.

### Which deep learning frameworks are used in the AI for Beginners NLP lessons?

The curriculum provides dual implementations for nearly every technique. **TensorFlow** notebooks (suffixed with `TF.ipynb`) use Keras APIs, while **PyTorch** notebooks (suffixed with `PyTorch.ipynb`) use `torch.nn` modules. This parallel structure allows learners to compare framework-specific implementations of identical algorithms.

### How does Lesson 18 explain Transformer architectures?

Rather than building Transformers from scratch, Lesson 18 focuses on practical application using the Hugging Face `transformers` library. It teaches learners to load pre-trained models, understand encoder-decoder attention mechanisms, and apply fine-tuning techniques to downstream tasks, bridging the gap between theory and production-ready implementation.

### What prompt engineering concepts are covered in Lesson 20?

Lesson 20 covers **prompt design strategies** for controlling GPT model outputs, along with **sampling techniques** including temperature scaling (controlling randomness), top-k sampling (restricting to k most likely tokens), and top-p/nucleus sampling (dynamic vocabulary truncation). The lesson also introduces beam search for optimal sequence generation and basic fine-tuning workflows.