How to Verify GPU Availability for PyTorch: Complete Detection Guide

Use torch.cuda.is_available() to check if PyTorch can access a CUDA-enabled GPU; returns True if the NVIDIA driver stack initializes successfully, False otherwise.

The Microsoft AI-For-Beginners repository demonstrates enterprise-grade GPU detection patterns across computer vision and natural language processing lessons. Verifying GPU availability ensures your PyTorch models leverage hardware acceleration when present while gracefully falling back to CPU execution.

The Canonical GPU Check in AI-For-Beginners

Every PyTorch-based lesson in the repository opens with the same sanity check. The pattern appears in lessons/4-ComputerVision/07-ConvNets/pytorchcv.py:

default_device = 'cuda' if torch.cuda.is_available() else 'cpu'

This one-liner queries the complete CUDA driver stack—NVIDIA kernel driver, CUDA runtime libraries, and GPU hardware—to confirm that device memory can be allocated and kernels launched.

How torch.cuda.is_available() Works

PyTorch executes a low-level initialization sequence when you call this function:

1. Loads the CUDA runtime libraries (libcudart.so on Linux, cudart.dll on Windows)
2. Enumerates available CUDA-capable devices
3. Attempts a minimal GPU context creation
4. Returns True only if all stages succeed

If any stage fails—missing drivers, incompatible CUDA version, or no hardware—the function returns False without raising an exception.

Device Selection Patterns Used in Production Code

Pattern 1: String-Based Device Selection

Found in lessons/4-ComputerVision/07-ConvNets/pytorchcv.py:

import torch

default_device = 'cuda' if torch.cuda.is_available() else 'cpu'
model = model.to(default_device)

Pattern 2: torch.device Object (Recommended)

Preferred in lessons/4-ComputerVision/08-TransferLearning/pytorchcv.py for explicit control:

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
inputs = inputs.to(device)
labels = labels.to(device)

Pattern 3: Jupyter Notebook Diagnostics

For interactive debugging, as implemented across NLP lessons like lessons/5-NLP/18-Transformers/torchnlp.py:

from torch import cuda

if cuda.is_available():
    print(f"CUDA version: {cuda.get_device_capability()}")
    print(f"GPU name: {cuda.get_device_name(0)}")
    print(f"Device count: {cuda.device_count()}")
else:
    print("No CUDA GPU detected — falling back to CPU")

Complete Verification Script

Run this diagnostic to fully characterize your PyTorch GPU environment:

import torch

def diagnose_gpu():
    """Comprehensive GPU availability check matching AI-For-Beginners patterns."""
    
    print(f"PyTorch version: {torch.__version__}")
    print(f"CUDA available: {torch.cuda.is_available()}")
    
    if torch.cuda.is_available():
        print(f"CUDA version: {torch.version.cuda}")
        print(f"cuDNN version: {torch.backends.cudnn.version()}")
        print(f"Device count: {torch.cuda.device_count()}")
        
        for i in range(torch.cuda.device_count()):
            print(f"  Device {i}: {torch.cuda.get_device_name(i)}")
            print(f"    Compute capability: {torch.cuda.get_device_capability(i)}")
            print(f"    Total memory: {torch.cuda.get_device_properties(i).total_memory / 1e9:.2f} GB")
        
        # Test actual tensor operations

        device = torch.device("cuda")
        test_tensor = torch.randn(1000, 1000, device=device)
        result = torch.mm(test_tensor, test_tensor.T)
        print(f"\nTest matmul on GPU: SUCCESS (shape {result.shape})")
    else:
        print("Running on CPU only")
        device = torch.device("cpu")

diagnose_gpu()

GPU Verification in Repository Lessons

The Microsoft AI-For-Beginners project applies consistent GPU detection across all deep learning modules:

Lesson	File Path	Detection Use Case
ConvNets	lessons/4-ComputerVision/07-ConvNets/pytorchcv.py	Convolutional network training with automatic device selection
Transfer Learning	lessons/4-ComputerVision/08-TransferLearning/pytorchcv.py	Fine-tuning pretrained models on available hardware
Embeddings	lessons/5-NLP/14-Embeddings/torchnlp.py	Word embedding computation with GPU acceleration
RNNs	lessons/5-NLP/16-RNN/torchnlp.py	Recurrent neural network training
Generative Networks	lessons/5-NLP/17-GenerativeNetworks/torchnlp.py	GAN-style generation with conditional CUDA usage
Transformers	lessons/5-NLP/18-Transformers/torchnlp.py	Attention-based models respecting GPU availability

Troubleshooting GPU Detection Failures

When torch.cuda.is_available() returns False, verify these components:

• NVIDIA Driver: Run nvidia-smi to confirm kernel driver is loaded
• CUDA Toolkit: Check nvcc --version matches PyTorch's CUDA build
• PyTorch Installation: Ensure you installed the CUDA-enabled wheel (torch not torch-cpu)
• Environment Variables: CUDA_VISIBLE_DEVICES may hide GPUs if misconfigured

# Quick system verification commands

nvidia-smi                    # GPU hardware and driver status

python -c "import torch; print(torch.version.cuda)"  # PyTorch CUDA build

python -c "import torch; print(torch.__file__)"      # PyTorch installation path

Summary

• Call torch.cuda.is_available() to programmatically detect GPU access before allocating tensors
• Use torch.device("cuda" if torch.cuda.is_available() else "cpu") for robust device selection across environments
• Reference pytorchcv.py and torchnlp.py in the AI-For-Beginners repository for production-tested patterns
• Verify the full stack: driver, runtime, and PyTorch build must align for successful GPU detection
• Always .to(device) both models and data to ensure operations execute on the selected hardware

Frequently Asked Questions

How do I check which GPU PyTorch is using?

Call torch.cuda.get_device_name(0) to retrieve the GPU model name, or iterate through all devices with torch.cuda.device_count(). For the currently active device, use torch.cuda.current_device().

Why does torch.cuda.is_available() return False when I have a GPU?

The most common causes are: PyTorch installed without CUDA support (install with pip install torch), missing or outdated NVIDIA drivers, or CUDA version mismatch between PyTorch build and system libraries. Run nvidia-smi to isolate driver issues from PyTorch configuration problems.

Does PyTorch automatically use GPU if available?

No—PyTorch defaults to CPU. You must explicitly move tensors and models to CUDA using .to(device) or specify device="cuda" during tensor creation. The AI-For-Beginners repository demonstrates this explicit pattern in every lesson.

Can I use multiple GPUs with this detection method?

Yes. After confirming availability with torch.cuda.is_available(), enumerate devices with torch.cuda.device_count() and distribute work using torch.nn.DataParallel or torch.distributed. The detection functions work identically for single and multi-GPU systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}