How to Verify GPU Detection in PyTorch and TensorFlow: Complete Guide with Code Examples

Both PyTorch and TensorFlow provide built-in APIs to detect GPU availability, allowing your code to automatically select the optimal device for training.

Verifying GPU detection is a critical first step in any deep learning workflow. The Microsoft AI-For-Beginners curriculum demonstrates practical patterns for checking GPU availability in both frameworks, ensuring your models run on accelerated hardware when possible. This guide walks through the exact methods used in the course materials, complete with source code references from the repository.

PyTorch GPU Detection with torch.cuda.is_available()

PyTorch uses a simple boolean check to determine CUDA availability. According to the AI-For-Beginners source code, the recommended pattern appears in lessons/5-NLP/16-RNN/torchnlp.py, lines 7–8:

import torch

# Detect GPU; fall back to CPU if none is found

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Using device: {device}")

This approach creates a reusable device object that you can pass to tensors and models throughout your training script. Calling torch.cuda.is_available() returns True when NVIDIA drivers and CUDA libraries are properly configured, and False otherwise.

Key PyTorch Detection Methods

• torch.cuda.is_available() – Returns boolean indicating CUDA runtime availability
• torch.cuda.device_count() – Returns the number of GPUs detected
• torch.cuda.get_device_name(0) – Returns the name of the specified GPU

TensorFlow GPU Detection with list_physical_devices

TensorFlow provides more granular device inspection through its configuration API. The RNN lesson notebook at translations/zh-TW/lessons/5-NLP/16-RNN/RNNTF.ipynb, line 54, demonstrates the standard pattern:

import tensorflow as tf

# List all GPUs TensorFlow can see

gpus = tf.config.list_physical_devices('GPU')
if gpus:
    print(f"TensorFlow detected {len(gpus)} GPU(s): {gpus}")
else:
    print("No GPU detected by TensorFlow; using CPU.")

Unlike PyTorch's boolean approach, TensorFlow returns a list of device objects. A non-empty list confirms GPU detection, while an empty list indicates CPU-only operation.

TensorFlow GPU Configuration Options

Method	Purpose
tf.config.list_physical_devices('GPU')	Enumerate all visible GPUs
tf.config.experimental.set_memory_growth()	Prevent TensorFlow from allocating all GPU memory
tf.config.set_visible_devices()	Restrict TensorFlow to specific GPUs

Complete GPU Verification Workflow

A robust training script should verify GPU availability before initializing models. Here's a unified pattern based on the AI-For-Beginners implementation:

import torch
import tensorflow as tf

def verify_pytorch_gpu():
    """Check PyTorch GPU status as implemented in torchnlp.py."""
    available = torch.cuda.is_available()
    device = torch.device("cuda" if available else "cpu")
    return {
        'available': available,
        'device': device,
        'count': torch.cuda.device_count() if available else 0
    }

def verify_tensorflow_gpu():
    """Check TensorFlow GPU status as shown in RNNTF.ipynb."""
    gpus = tf.config.list_physical_devices('GPU')
    return {
        'available': len(gpus) > 0,
        'devices': gpus,
        'count': len(gpus)
    }

# Verify both frameworks

print("PyTorch GPU check:", verify_pytorch_gpu())
print("TensorFlow GPU check:", verify_tensorflow_gpu())

Troubleshooting GPU Detection Failures

When torch.cuda.is_available() returns False or tf.config.list_physical_devices('GPU') returns [], verify these common causes:

• NVIDIA drivers – Install the latest drivers for your GPU model
• CUDA toolkit version – Ensure compatibility with your framework version
• cuDNN libraries – Required for deep learning operations on NVIDIA GPUs
• Framework installation – Reinstall PyTorch or TensorFlow with GPU support: pip install torch or pip install tensorflow[and-cuda]

Summary

• PyTorch GPU detection – Use torch.cuda.is_available() to return a boolean, then create a torch.device() object for tensor allocation, as demonstrated in lessons/5-NLP/16-RNN/torchnlp.py
• TensorFlow GPU detection – Use tf.config.list_physical_devices('GPU') to return a list of device objects, checking list length to confirm availability, as shown in translations/zh-TW/lessons/5-NLP/16-RNN/RNNTF.ipynb
• Both methods enable automatic CPU fallback, preventing runtime errors when GPU acceleration is unavailable
• Store the detected device early in your script and reuse it across all tensor and model operations

Frequently Asked Questions

How do I check if PyTorch is using GPU after creating tensors?

Call tensor.is_cuda on any PyTorch tensor to verify its device placement, or use tensor.device to see the specific device name. For models, next(model.parameters()).device reveals where parameters reside.

Why does TensorFlow detect my GPU but PyTorch does not?

This typically indicates version mismatches in CUDA or cuDNN installations. TensorFlow and PyTorch often require different CUDA versions—check each framework's compatibility matrix and consider using separate conda environments.

Can I force GPU detection to fail for testing CPU code paths?

Yes. In PyTorch, set CUDA_VISIBLE_DEVICES="" before import to hide all GPUs. In TensorFlow, call tf.config.set_visible_devices([], 'GPU') to disable GPU visibility at runtime.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}