# How to Use Prompt Programming with Large Language Models for Few-Shot Learning

> Master prompt programming for few-shot learning with large language models. Learn to guide LLMs to infer patterns and make predictions with structured prompts and examples without fine-tuning.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: how-to-guide
- Published: 2026-08-09

---

**Prompt programming with large language models for few-shot learning involves crafting a structured text prompt that includes a task description, several labeled examples, and a new input, enabling pre-trained transformers to infer patterns and generate accurate predictions without gradient updates or fine-tuning.**

The microsoft/AI-For-Beginners repository explains how generative pre-trained transformers (GPT) such as GPT-2, GPT-3, and GPT-4 leverage their pre-training on massive corpora to perform downstream tasks through careful prompt design. By utilizing conditional probability distributions learned during pre-training, these models can extrapolate from as few as one to five examples to classify sentiment, translate languages, or answer questions.

## Understanding Few-Shot Learning and Prompt Programming

Large language models learn to predict the next token in a sequence during pre-training, capturing the conditional probability \(P(w_N\mid w_{n-1},\dots,w_0)\) as described in [`lessons/5-NLP/20-LangModels/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/20-LangModels/README.md) (lines 13-14). This fundamental mechanism, referenced in the seminal paper *Language Models are Unsupervised Multitask Learners*, allows models to continue any coherent text pattern presented in a prompt.

### The Few-Shot Inference Mechanism

According to the source code analysis at line 51 of [`lessons/5-NLP/20-LangModels/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/20-LangModels/README.md), LLMs can achieve reasonable performance on new tasks by extrapolating from 1–5 examples embedded directly in the prompt. The typical few-shot prompt structure includes:

- **Task description**: A clear instruction defining the objective (e.g., "Classify the sentiment")
- **Labeled examples**: Input-output pairs demonstrating the desired pattern
- **Target input**: The new instance requiring prediction, typically ending with the expected label token to guide completion

As noted in lines 38-41 of the README, the quality of results depends heavily on **prompt engineering**—the strategic selection of wording, format, and delimiters to guide the model toward intended behavior.

## Implementing Few-Shot Prompting in Python

### Local Inference with HuggingFace Transformers

For development environments requiring local execution, you can implement few-shot learning using the `transformers` library. The `lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb` notebook demonstrates this pattern with PyTorch:

```python
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Load a pre-trained GPT-2 model

tokenizer = GPT2Tokenizer.from_pretrained("gpt2-large")
model = GPT2LMHeadModel.from_pretrained("gpt2-large")
model.eval()

# Build a few-shot prompt for sentiment analysis

prompt = """Classify the sentiment of the following sentences as Positive or Negative.

Sentence: I love this movie. Sentiment: Positive
Sentence: The food was terrible. Sentiment: Negative
Sentence: The plot was thrilling. Sentiment:"""

# Tokenize and generate

input_ids = tokenizer.encode(prompt, return_tensors="pt")
with torch.no_grad():
    output = model.generate(
        input_ids,
        max_new_tokens=10,
        temperature=0.7,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
    )
generated = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated.split("Sentiment:")[-1].strip())

```

This example embeds two labeled sentiment examples before the target sentence. The `model.generate()` method continues the pattern, using `temperature=0.7` to control creativity while maintaining coherence.

### Cloud-Based Prompting with Azure OpenAI

For production deployments, Azure OpenAI Service provides managed endpoints that accept the same few-shot format. The implementation uses the `openai` Python library with Azure-specific configuration:

```python
import os
import openai

# Azure endpoint configuration

openai.api_type = "azure"
openai.api_base = os.getenv("AZURE_OPENAI_ENDPOINT")
openai.api_version = "2023-05-15"
openai.api_key = os.getenv("AZURE_OPENAI_KEY")

deployment_name = "gpt-35-turbo"  # Azure deployment name

few_shot_prompt = """You are a helpful assistant that classifies movie reviews.

Review: "The cinematography was stunning." -> Sentiment: Positive
Review: "I fell asleep halfway through." -> Sentiment: Negative
Review: "The storyline kept me engaged." -> Sentiment:"""

response = openai.ChatCompletion.create(
    engine=deployment_name,
    messages=[{"role": "user", "content": few_shot_prompt}],
    temperature=0.6,
    max_tokens=10,
)

print(response.choices[0].message["content"].strip())

```

The `ChatCompletion.create()` method processes the few-shot prompt through Azure's managed infrastructure, returning the predicted label without requiring local GPU resources.

## Prompt Engineering Best Practices

Effective prompt programming requires iterative refinement of the input format. Based on the architecture described in the microsoft/AI-For-Beginners materials, optimize your prompts using these strategies:

- **Use explicit delimiters** (e.g., `=== Example ===` or `->`) to clearly separate input-output pairs and prevent pattern confusion
- **Maintain balanced examples** by including both positive and negative class instances to avoid bias in the model's predictions
- **Terminate with the label token** by ending the prompt with the target input followed immediately by the expected output key (e.g., `Sentiment:`), forcing the model to complete with the category rather than additional text
- **Adjust generation parameters**: Lower `temperature` values (0.1-0.3) increase determinism for classification tasks, while higher values (0.7-0.9) improve creativity for open-ended generation

## Summary

- **Prompt programming** leverages pre-trained LLMs' ability to model conditional probabilities, eliminating the need for task-specific fine-tuning
- **Few-shot learning** requires only 1-5 labeled examples embedded directly in the prompt, as implemented in [`lessons/5-NLP/20-LangModels/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/20-LangModels/README.md)
- **Local execution** is possible using HuggingFace Transformers with `GPT2LMHeadModel` and appropriate tokenization
- **Cloud deployment** via Azure OpenAI Service utilizes the `ChatCompletion.create()` endpoint with identical few-shot formatting
- **Prompt engineering** quality—encompassing delimiter choice, example ordering, and temperature settings—directly impacts model performance

## Frequently Asked Questions

### What is the difference between zero-shot and few-shot prompting?

Zero-shot prompting provides only a task description and the target input without examples, relying entirely on the model's pre-trained knowledge. Few-shot prompting includes 1-5 labeled examples in the prompt context, allowing the model to recognize specific patterns or formatting requirements for the target task. According to the microsoft/AI-For-Beginners source, few-shot approaches typically yield higher accuracy for specialized classification tasks.

### How many examples are needed for effective few-shot learning?

Most GPT-style models achieve reasonable performance with as few as one to five examples, as noted at line 51 of [`lessons/5-NLP/20-LangModels/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/20-LangModels/README.md). The exact number depends on task complexity: simple binary classification may require only 2-3 balanced examples, while multi-class or structured output tasks may benefit from 5-10 diverse examples. Adding too many examples risks exceeding the model's context window or diluting the pattern.

### Can prompt programming replace fine-tuning for production applications?

Prompt programming is ideal for rapid prototyping, low-data scenarios, or tasks requiring quick adaptation without computational overhead. However, for applications requiring consistent formatting across thousands of diverse inputs or specialized domain knowledge not present in the base model, fine-tuning may provide more robust performance. The microsoft/AI-For-Beginners materials suggest using prompt engineering as the first approach before considering fine-tuning.

### Which open-source models support few-shot prompt programming?

While the examples in `lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb` focus on GPT-2, modern alternatives include GPT-J, GPT-NeoX, LLaMA 2, and Mistral models available through HuggingFace. These models support identical few-shot prompting techniques using the `AutoModelForCausalLM` and `AutoTokenizer` classes, though their performance may vary based on model size and training data diversity compared to GPT-3 or GPT-4.