How to Enable GPU Support for TensorFlow and PyTorch in the AI-For-Beginners Curriculum

Install NVIDIA drivers and CUDA, modify environment.yml to use CUDA-compatible PyTorch builds, and verify GPU detection with tf.config.list_physical_devices('GPU') or torch.cuda.is_available() before running any notebook.

The Microsoft AI-For-Beginners curriculum provides hands-on deep learning lessons through Jupyter notebooks using TensorFlow and PyTorch. While both frameworks automatically utilize GPUs when available, enabling GPU support requires proper driver installation, CUDA toolkit configuration, and environment adjustments to the repository's default CPU-focused setup.

Understanding the GPU Architecture Stack

The curriculum interacts with GPU hardware through several layers. Knowing this stack helps diagnose configuration issues quickly.

Layer	Components	Curriculum Interaction
Hardware	NVIDIA GPU with driver ≥ 450	Provides CUDA cores for tensor operations
System Drivers	NVIDIA driver, CUDA toolkit, cuDNN	Queried at import time by both frameworks
Python Packages	tensorflow, pytorch, cudatoolkit	Installed via environment.yml or manual override
Notebook Runtime	Jupyter, VS Code, Azure, Colab	Executes framework code without hard-coded device selection
Framework Config	Memory growth settings, device selection	Optional snippets in translated READMEs

The notebooks in lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb and similar paths automatically detect and use GPUs without code changes—provided the environment is correctly configured.

Step 1: Install NVIDIA Drivers and CUDA

Before modifying any Python environment, verify your system has compatible GPU drivers.

1. Download drivers from NVIDIA's official site matching your GPU model
2. Install CUDA toolkit 11.8 or 12.x (TensorFlow 2.17+ supports CUDA 12)
3. Install cuDNN corresponding to your CUDA version

Confirm installation with:

nvidia-smi
nvcc --version

Both commands should execute without errors and display version information.

Step 2: Configure the Conda Environment for GPU

The repository's environment.yml at the root defaults to CPU-only packages. You must recreate the environment with CUDA-compatible builds.

Create and activate the base environment:

conda env create -f environment.yml
conda activate ai4beg

Then reinstall PyTorch with explicit CUDA support. For CUDA 12.1:

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

For TensorFlow, upgrade to a GPU-enabled version:

pip install --upgrade "tensorflow>=2.17"

TensorFlow 2.x automatically detects CUDA installations when drivers are properly configured. No separate tensorflow-gpu package is needed.

Step 3: Verify GPU Detection

Always validate GPU availability before running lessons. Create a verification script or run these commands in a notebook:

import tensorflow as tf
print("TensorFlow GPUs:", tf.config.list_physical_devices('GPU'))

import torch
print("PyTorch CUDA available:", torch.cuda.is_available())
print("PyTorch GPU count:", torch.cuda.device_count())
print("PyTorch GPU name:", torch.cuda.get_device_name(0) if torch.cuda.is_available() else "None")

Both frameworks should report at least one GPU device. Empty lists or False values indicate driver or package misconfiguration.

Step 4: Configure TensorFlow GPU Memory Growth

TensorFlow's default behavior allocates all GPU memory at startup, which can cause out-of-memory errors when running multiple models. The curriculum's translated documentation in lessons/5-NLP/README.md (line 48) recommends enabling dynamic memory growth.

Add this snippet at the start of any TensorFlow notebook:

import tensorflow as tf

gpus = tf.config.list_physical_devices('GPU')
if gpus:
    try:
        for gpu in gpus:
            tf.config.experimental.set_memory_growth(gpu, True)
        print("GPU memory growth enabled")
    except Exception as e:
        print("Failed to set memory growth:", e)

This prevents TensorFlow from monopolizing VRAM, allowing multiple notebook cells or concurrent processes to share the GPU.

Step 5: Explicit Device Selection in PyTorch

Unlike TensorFlow, PyTorch requires explicit tensor and model placement on the GPU. The curriculum notebooks follow this pattern, as seen in lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb:

import torch
import torch.nn as nn

# Select device

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")

# Define and move model

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(784, 10)

    def forward(self, x):
        return self.fc(x)

model = Net().to(device)

# Create data directly on GPU

x = torch.randn(64, 784, device=device)
output = model(x)

The .to(device) call is mandatory—PyTorch will not automatically move existing tensors to the GPU even when one is available.

Cloud GPU Alternatives

If local GPU configuration proves difficult, the curriculum documentation in AGENTS.md (lines 253-257) and lessons/0-course-setup/how-to-run.md suggests several cloud options:

• Azure Data Science VM (NC-series): Pre-configured with drivers and frameworks
• Azure Machine Learning: GPU-enabled compute clusters attachable to notebooks
• Google Colab: Free Tesla T4 GPUs; upload notebooks directly with Runtime > Change runtime type > GPU

These environments typically require no additional setup—GPU detection works immediately upon import.

Troubleshooting Common GPU Issues

Symptom	Diagnosis	Solution
[] from tf.config.list_physical_devices('GPU')	Missing driver, CUDA mismatch, or CPU-only packages	Reinstall NVIDIA driver; verify nvcc --version; reinstall PyTorch with matching cudatoolkit
torch.cuda.is_available() returns False	Same as above, or PyTorch installed without CUDA	Run conda install command with explicit pytorch-cuda channel
Out-of-memory during training	Batch size exceeds VRAM	Reduce batch_size parameter; enable TensorFlow memory growth
"No CUDA runtime is found"	Environment variables missing CUDA paths	Add CUDA bin and lib directories to PATH and LD_LIBRARY_PATH
Computations run on CPU despite cuda available	Model/tensors not moved to device	Verify .to(device) calls on all tensors and model outputs

Summary

• Install NVIDIA drivers, CUDA toolkit, and cuDNN before any Python configuration
• Recreate the Conda environment from environment.yml, then override with CUDA-specific PyTorch builds
• Verify GPU detection with framework-specific commands before running lessons
• Enable TensorFlow memory growth to prevent VRAM exhaustion in multi-model notebooks
• Explicitly move PyTorch models and tensors to 'cuda' device—no automatic placement occurs
• Use cloud GPU instances (Azure DSVM, Colab) when local setup is impractical

Frequently Asked Questions

Does the AI-For-Beginners curriculum require GPU support to complete the lessons?

No. All notebooks execute correctly on CPU, though training times increase significantly for deep learning models. The environment.yml defaults to CPU-only packages specifically for accessibility. GPU support is optional but recommended for lessons involving convolutional networks and transformers in lessons/4-ComputerVision and lessons/5-NLP.

Why does PyTorch require .to(device) while TensorFlow works automatically?

PyTorch follows an explicit programming model where developers control device placement for debugging and multi-GPU flexibility. TensorFlow adopts automatic device placement with optional constraints. The curriculum respects these design differences—PyTorch notebooks include device handling code, while TensorFlow notebooks run unchanged on GPU when available.

Can I use AMD GPUs or Apple Silicon with this curriculum?

Partial support exists. Apple Silicon Macs can use PyTorch's MPS backend (torch.device('mps')), though this requires manual code changes to existing notebooks. AMD GPUs through ROCm work with specific PyTorch builds but are not documented in the official curriculum. NVIDIA CUDA remains the primary supported platform as referenced in lessons/0-course-setup/how-to-run.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}