# How to Enable GPU Support for TensorFlow and PyTorch in the AI-For-Beginners Curriculum

> Accelerate AI learning by enabling GPU support for TensorFlow and PyTorch in the AI-For-Beginners curriculum. Follow simple steps to install drivers, configure environment files, and verify GPU detection for faster model training.

- Repository: [Microsoft/AI-For-Beginners](https://github.com/microsoft/AI-For-Beginners)
- Tags: how-to-guide
- Published: 2026-08-18

---

**Install NVIDIA drivers and CUDA, modify [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) to use CUDA-compatible PyTorch builds, and verify GPU detection with `tf.config.list_physical_devices('GPU')` or `torch.cuda.is_available()` before running any notebook.**

The Microsoft **AI-For-Beginners** curriculum provides hands-on deep learning lessons through Jupyter notebooks using TensorFlow and PyTorch. While both frameworks automatically utilize GPUs when available, enabling GPU support requires proper driver installation, CUDA toolkit configuration, and environment adjustments to the repository's default CPU-focused setup.

## Understanding the GPU Architecture Stack

The curriculum interacts with GPU hardware through several layers. Knowing this stack helps diagnose configuration issues quickly.

| Layer | Components | Curriculum Interaction |
|-------|-----------|------------------------|
| **Hardware** | NVIDIA GPU with driver ≥ 450 | Provides CUDA cores for tensor operations |
| **System Drivers** | NVIDIA driver, CUDA toolkit, cuDNN | Queried at import time by both frameworks |
| **Python Packages** | `tensorflow`, `pytorch`, `cudatoolkit` | Installed via [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) or manual override |
| **Notebook Runtime** | Jupyter, VS Code, Azure, Colab | Executes framework code without hard-coded device selection |
| **Framework Config** | Memory growth settings, device selection | Optional snippets in translated READMEs |

The notebooks in `lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb` and similar paths automatically detect and use GPUs without code changes—provided the environment is correctly configured.

## Step 1: Install NVIDIA Drivers and CUDA

Before modifying any Python environment, verify your system has compatible GPU drivers.

1. Download drivers from [NVIDIA's official site](https://www.nvidia.com/drivers) matching your GPU model
2. Install CUDA toolkit 11.8 or 12.x (TensorFlow 2.17+ supports CUDA 12)
3. Install cuDNN corresponding to your CUDA version

Confirm installation with:

```bash
nvidia-smi
nvcc --version

```

Both commands should execute without errors and display version information.

## Step 2: Configure the Conda Environment for GPU

The repository's [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) at the root defaults to CPU-only packages. You must recreate the environment with CUDA-compatible builds.

Create and activate the base environment:

```bash
conda env create -f environment.yml
conda activate ai4beg

```

Then reinstall PyTorch with explicit CUDA support. For CUDA 12.1:

```bash
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

```

For TensorFlow, upgrade to a GPU-enabled version:

```bash
pip install --upgrade "tensorflow>=2.17"

```

TensorFlow 2.x automatically detects CUDA installations when drivers are properly configured. No separate `tensorflow-gpu` package is needed.

## Step 3: Verify GPU Detection

Always validate GPU availability before running lessons. Create a verification script or run these commands in a notebook:

```python
import tensorflow as tf
print("TensorFlow GPUs:", tf.config.list_physical_devices('GPU'))

```

```python
import torch
print("PyTorch CUDA available:", torch.cuda.is_available())
print("PyTorch GPU count:", torch.cuda.device_count())
print("PyTorch GPU name:", torch.cuda.get_device_name(0) if torch.cuda.is_available() else "None")

```

Both frameworks should report at least one GPU device. Empty lists or `False` values indicate driver or package misconfiguration.

## Step 4: Configure TensorFlow GPU Memory Growth

TensorFlow's default behavior allocates all GPU memory at startup, which can cause out-of-memory errors when running multiple models. The curriculum's translated documentation in [`lessons/5-NLP/README.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/README.md) (line 48) recommends enabling dynamic memory growth.

Add this snippet at the start of any TensorFlow notebook:

```python
import tensorflow as tf

gpus = tf.config.list_physical_devices('GPU')
if gpus:
    try:
        for gpu in gpus:
            tf.config.experimental.set_memory_growth(gpu, True)
        print("GPU memory growth enabled")
    except Exception as e:
        print("Failed to set memory growth:", e)

```

This prevents TensorFlow from monopolizing VRAM, allowing multiple notebook cells or concurrent processes to share the GPU.

## Step 5: Explicit Device Selection in PyTorch

Unlike TensorFlow, PyTorch requires explicit tensor and model placement on the GPU. The curriculum notebooks follow this pattern, as seen in `lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb`:

```python
import torch
import torch.nn as nn

# Select device

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")

# Define and move model

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(784, 10)

    def forward(self, x):
        return self.fc(x)

model = Net().to(device)

# Create data directly on GPU

x = torch.randn(64, 784, device=device)
output = model(x)

```

The `.to(device)` call is mandatory—PyTorch will not automatically move existing tensors to the GPU even when one is available.

## Cloud GPU Alternatives

If local GPU configuration proves difficult, the curriculum documentation in [`AGENTS.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/AGENTS.md) (lines 253-257) and [`lessons/0-course-setup/how-to-run.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/0-course-setup/how-to-run.md) suggests several cloud options:

- **Azure Data Science VM (NC-series)**: Pre-configured with drivers and frameworks
- **Azure Machine Learning**: GPU-enabled compute clusters attachable to notebooks
- **Google Colab**: Free Tesla T4 GPUs; upload notebooks directly with `Runtime > Change runtime type > GPU`

These environments typically require no additional setup—GPU detection works immediately upon import.

## Troubleshooting Common GPU Issues

| Symptom | Diagnosis | Solution |
|---------|-----------|----------|
| `[]` from `tf.config.list_physical_devices('GPU')` | Missing driver, CUDA mismatch, or CPU-only packages | Reinstall NVIDIA driver; verify `nvcc --version`; reinstall PyTorch with matching `cudatoolkit` |
| `torch.cuda.is_available()` returns `False` | Same as above, or PyTorch installed without CUDA | Run `conda install` command with explicit `pytorch-cuda` channel |
| Out-of-memory during training | Batch size exceeds VRAM | Reduce `batch_size` parameter; enable TensorFlow memory growth |
| "No CUDA runtime is found" | Environment variables missing CUDA paths | Add CUDA `bin` and `lib` directories to `PATH` and `LD_LIBRARY_PATH` |
| Computations run on CPU despite `cuda` available | Model/tensors not moved to device | Verify `.to(device)` calls on all tensors and model outputs |

## Summary

- Install NVIDIA drivers, CUDA toolkit, and cuDNN before any Python configuration
- Recreate the Conda environment from [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml), then override with CUDA-specific PyTorch builds
- Verify GPU detection with framework-specific commands before running lessons
- Enable TensorFlow memory growth to prevent VRAM exhaustion in multi-model notebooks
- Explicitly move PyTorch models and tensors to `'cuda'` device—no automatic placement occurs
- Use cloud GPU instances (Azure DSVM, Colab) when local setup is impractical

## Frequently Asked Questions

### Does the AI-For-Beginners curriculum require GPU support to complete the lessons?

No. All notebooks execute correctly on CPU, though training times increase significantly for deep learning models. The [`environment.yml`](https://github.com/microsoft/AI-For-Beginners/blob/main/environment.yml) defaults to CPU-only packages specifically for accessibility. GPU support is optional but recommended for lessons involving convolutional networks and transformers in `lessons/4-ComputerVision` and `lessons/5-NLP`.

### Why does PyTorch require `.to(device)` while TensorFlow works automatically?

PyTorch follows an explicit programming model where developers control device placement for debugging and multi-GPU flexibility. TensorFlow adopts automatic device placement with optional constraints. The curriculum respects these design differences—PyTorch notebooks include device handling code, while TensorFlow notebooks run unchanged on GPU when available.

### Can I use AMD GPUs or Apple Silicon with this curriculum?

Partial support exists. Apple Silicon Macs can use PyTorch's MPS backend (`torch.device('mps')`), though this requires manual code changes to existing notebooks. AMD GPUs through ROCm work with specific PyTorch builds but are not documented in the official curriculum. NVIDIA CUDA remains the primary supported platform as referenced in [`lessons/0-course-setup/how-to-run.md`](https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/0-course-setup/how-to-run.md).