How to Fix torch.cuda.is_available() Returning False in NVIDIA Cosmos

TLDR: In the NVIDIA/cosmos repository, torch.cuda.is_available() returns False when the CUDA build of PyTorch installed by uv does not match your NVIDIA driver's CUDA version; fix it by reinstalling PyTorch with the matching --torch-backend flag (e.g., cu128 for CUDA 12.8 or cu130 for CUDA 13.0).

Cosmos relies on a uv-managed virtual environment and a PyTorch wheel compiled for a specific CUDA backend. When that wheel's CUDA version diverges from the host driver, the framework silently loses GPU acceleration. If you are troubleshooting torch.cuda.is_available() returning False with Cosmos, the root cause is almost always this backend mismatch, as documented in the project's README and cookbook notebooks.

Root Cause: Why torch.cuda.is_available() Returns False in Cosmos

Cosmos installs PyTorch through uv using the --torch-backend flag. As noted in README.md (lines 31-34 and 90-94), uv defaults to the newest CUDA wheel (currently cu130), which fails on machines with older drivers such as CUDA 12.8. When you import torch, it compares torch.version.cuda against the driver runtime. If the versions differ, torch.cuda.is_available() evaluates to False and the server falls back to CPU or fails to start.

Diagnose the torch.cuda.is_available() Mismatch

Before reinstalling, confirm the CUDA version your driver actually supports. Run the diagnostic command shown in the Cosmos cookbooks:

nvidia-smi

Note the CUDA version listed in the top-right corner (for example, 12.8 or 13.0). This major version must match the uv backend you select.

The notebook cookbooks/cosmos3/reasoner/run_with_cosmos_framework.ipynb (lines 249-255) prints a standard verification block that you can use to compare the driver against PyTorch at runtime.

Reinstall PyTorch with the Correct --torch-backend

Step 1: Identify the Correct Backend Flag

Map your driver CUDA version to the uv flag:

• CUDA 13.x drivers → --torch-backend=cu130
• CUDA 12.8 drivers → --torch-backend=cu128

This mapping is listed in README.md (lines 31-34) and explicitly paired with vLLM versions in cookbooks/cosmos3/reasoner/run_with_vllm.ipynb (lines 39-42).

Step 2: Reinstall the Matching Wheel

Inside your existing .venv, uninstall the mismatched build and install the correct one:

uv pip uninstall torch torchvision
uv pip install --torch-backend=cu128 torch torchvision

Replace cu128 with cu130 if your driver supports CUDA 13.

Step 3: Verify GPU Access

Run the Python diagnostic snippet used across Cosmos notebooks:

import torch

print("torch version:", torch.__version__)
print("torch CUDA:", torch.version.cuda)
print("CUDA available:", torch.cuda.is_available())
print("GPU count:", torch.cuda.device_count())
print("GPU name:", torch.cuda.get_device_name(0))

When the installation is correct, torch.cuda.is_available() returns True and torch.version.cuda matches your nvidia-smi output.

Automate the Fix with a Check-and-Install Script

You can automate detection and reinstallation. The following script reads the driver CUDA version and invokes uv with the proper backend:

#!/usr/bin/env bash
set -euo pipefail

DRIVER_CUDA=$(nvidia-smi --query-gpu=cuda_version --format=csv,noheader | head -n1)
echo "Driver reports CUDA $DRIVER_CUDA"

case "${DRIVER_CUDA}" in
  13.*) BACKEND="cu130" ;;
  12.8*) BACKEND="cu128" ;;
  *) echo "Unsupported CUDA version $DRIVER_CUDA" >&2; exit 1 ;;
esac

echo "Using uv torch backend: $BACKEND"

uv venv --python 3.13 --seed --managed-python
source .venv/bin/activate
uv pip install --torch-backend=$BACKEND torch torchvision

Running this guarantees that torch.version.cuda aligns with the driver, eliminating the False return from torch.cuda.is_available().

Why --torch-backend=auto Fails in Cosmos

uv supports --torch-backend=auto, which attempts to infer the correct wheel from the host driver. However, as highlighted in README.md (lines 31-34) and cookbooks/cosmos3/generator/audiovisual/run_with_diffusers.ipynb, vLLM does not publish wheels for every minor CUDA version. Auto-detection can therefore pull a wheel that is newer than your driver, reproducing the same torch.cuda.is_available() == False symptom. The Cosmos documentation recommends pinning the backend explicitly instead of relying on auto.

Summary

• torch.cuda.is_available() returns False in Cosmos when the uv-installed PyTorch wheel targets a different CUDA version than your NVIDIA driver.
• Check your driver with nvidia-smi, then map it to --torch-backend=cu128 (CUDA 12.8) or --torch-backend=cu130 (CUDA 13).
• Reinstall torch and torchvision with the matching flag inside your .venv.
• Avoid --torch-backend=auto because vLLM wheel gaps can cause it to select an incompatible build.
• Confirm the fix with the diagnostic prints in cookbooks/cosmos3/reasoner/run_with_cosmos_framework.ipynb.

Frequently Asked Questions

Why does torch.cuda.is_available() return False even though nvidia-smi works?

nvidia-smi shows the driver capability, not the CUDA runtime linked to PyTorch. If uv installed a cu130 wheel but your driver only supports CUDA 12.8, PyTorch cannot initialize the CUDA context and returns False. Reinstall with --torch-backend=cu128 to match the driver.

Can I use --torch-backend=auto to fix the mismatch?

No. According to the Cosmos source documentation in README.md and the audiovisual cookbook, auto can select a vLLM wheel that is newer than your driver because vLLM does not publish wheels for every minor release. Pin the backend manually to ensure compatibility.

Which Cosmos files document the correct backend to vLLM pairing?

The pairing is documented in cookbooks/cosmos3/reasoner/run_with_vllm.ipynb (lines 39-42), where specific backend flags are matched to vLLM versions. The main troubleshooting steps appear in README.md (lines 90-94).

How do I verify the fix inside a notebook?

Execute the verification block from cookbooks/cosmos3/reasoner/run_with_cosmos_framework.ipynb (lines 249-255). It prints torch.version.cuda and torch.cuda.is_available(). If the CUDA version string matches your driver and the availability flag is True, Cosmos can access the GPU.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}