# How WhichLLM Detects GPU Hardware: Cross-Platform GPU Detection Explained

> Discover how WhichLLM detects GPU hardware on Linux, macOS, and Windows using a unified, OS-aware system. Get robust cross-platform hardware info.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: internals
- Published: 2026-06-10

---

**WhichLLM detects GPU hardware using a modular, OS-aware orchestration system that aggregates vendor-specific detectors into a unified `HardwareInfo` object, ensuring robust cross-platform compatibility across Linux, macOS, and Windows.**

The `whichllm` repository implements a sophisticated hardware detection layer designed to identify GPUs from NVIDIA, AMD, Intel, and Apple across different operating systems. By combining native system tools, Python bindings, and hardware databases, the detection system provides detailed GPU specifications including VRAM, compute capability, and memory bandwidth. This architecture ensures that WhichLLM can recommend optimal large language models based on your actual hardware capabilities.

## The Detection Orchestrator

At the heart of WhichLLM's GPU detection lies the `detect_hardware()` function in [`src/whichllm/hardware/detector.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/detector.py). This central orchestrator determines the host operating system using `platform.system()` and dispatches to appropriate vendor-specific detectors based on the platform.

### OS Detection and Dispatch Logic

The orchestrator implements a prioritized detection strategy:

- **NVIDIA** – Always queried regardless of OS via `detect_nvidia_gpus()`
- **Linux** – Checks AMD (`detect_amd_gpus`), Intel (`detect_intel_gpus`), and Apple-on-Linux (`detect_apple_gpu_linux`)
- **macOS** – Queries Apple silicon via `detect_apple_gpu()`
- **Windows** – Uses generic Windows GPU detection (`detect_windows_gpus`) for non-NVIDIA hardware

Each detector operates independently and returns a list of GPU objects, which the orchestrator aggregates into the final `HardwareInfo` result.

## Vendor-Specific Detection Strategies

### NVIDIA GPU Detection (NVML and nvidia-smi)

The `detect_nvidia_gpus()` function in [`src/whichllm/hardware/nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/nvidia.py) implements a two-tier fallback system. It first attempts to use the **NVML** Python bindings (`pynvml`) for direct hardware communication. If NVML initialization fails or the library is unavailable, it falls back to parsing the **`nvidia-smi`** CLI output.

For each detected device, the module collects:
- GPU name and VRAM capacity
- CUDA driver version
- Compute capability (via `NVIDIA_COMPUTE_CAPABILITY` lookup)
- Memory bandwidth (via `resolve_detected_bandwidth`)

The system also handles unified-memory GPUs such as the "GB10" and "DGX SPARK" by treating system RAM as VRAM when dedicated video memory is not present.

### AMD GPU Detection (rocm-smi and sysfs)

AMD detection in [`src/whichllm/hardware/amd.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/amd.py) follows a similar dual-path approach. The primary method executes **`rocm-smi`** with JSON output formatting to extract GPU specifications. When `rocm-smi` is unavailable, the detector falls back to reading sysfs DRM entries in `/sys/class/drm` to obtain vendor IDs, device IDs, and VRAM information directly from the kernel.

### Intel Integrated GPU Detection (lspci)

For Intel graphics, the `detect_intel_gpus()` function in [`src/whichllm/hardware/intel.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/intel.py) executes **`lspci -nn -k`** and parses the output for Intel graphics devices. It extracts VRAM specifications where reported by the hardware, making it compatible with both integrated and discrete Intel GPUs on Linux systems.

### Apple GPU Detection (macOS and Linux)

Apple hardware detection splits across two functions in [`src/whichllm/hardware/apple.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/apple.py):

- **macOS**: `detect_apple_gpu()` parses output from `system_profiler SPDisplaysDataType` to obtain GPU names and memory configurations
- **Linux**: `detect_apple_gpu_linux()` handles Asahi driver installations by reading `/sys/class/drm` entries, similar to the AMD sysfs fallback method

### Windows Non-NVIDIA Detection (WMI)

The `detect_windows_gpus()` function in [`src/whichllm/hardware/windows.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/windows.py) utilizes **WMI** (`win32com.client`) to query the `Win32_VideoController` class. This extracts GPU names, RAM amounts, driver versions, and vendor information for non-NVIDIA hardware on Windows systems. NVIDIA detection on Windows is delegated to the dedicated NVIDIA module to avoid duplicate entries.

## Memory Bandwidth Resolution

After detecting GPU hardware, WhichLLM resolves memory bandwidth specifications through the `resolve_detected_bandwidth()` function in [`src/whichllm/hardware/gpu_db.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/gpu_db.py). This function first checks a curated `GPU_BANDWIDTH` lookup table for known hardware configurations. If the GPU is not found in the local table, it falls back to querying the **`dbgpu`** database for comprehensive coverage of modern and legacy graphics cards.

## Fail-Safe Architecture

All detector functions in WhichLLM implement fail-safe error handling. Each detector wraps its execution in exception handlers that catch any errors, log the failure, and return an empty list rather than raising exceptions. This design ensures that a failure to detect one GPU vendor never aborts the entire hardware scan, allowing WhichLLM to provide partial hardware information even when specific drivers or tools are missing.

## Practical Usage Examples

To run the complete hardware detection stack in your Python application:

```python
from whichllm.hardware.detector import detect_hardware

# Run the orchestrated detection

info = detect_hardware()

# Print a summary of detected GPUs

for gpu in info.gpus:
    print(f"GPU: {gpu.name}")
    print(f"  Vendor: {gpu.vendor}")
    print(f"  VRAM: {gpu.vram_bytes // (1024**3)} GiB")
    print(f"  Compute Capability: {gpu.compute_capability}")
    print(f"  Memory Bandwidth: {gpu.memory_bandwidth_gbps} GB/s")
    print(f"  Unified Memory: {gpu.shared_memory}")
    print()

```

For targeted NVIDIA detection without running the full orchestrator:

```python
from whichllm.hardware.nvidia import detect_nvidia_gpus

nvidia_gpus = detect_nvidia_gpus()
if nvidia_gpus:
    print("Detected NVIDIA GPUs:", [g.name for g in nvidia_gpus])
else:
    print("No NVIDIA GPUs found.")

```

## Summary

- **WhichLLM detects GPU hardware** through a centralized orchestrator in [`detector.py`](https://github.com/Andyyyy64/whichllm/blob/main/detector.py) that dispatches OS-specific detection routines.
- The system supports **NVIDIA** (via NVML/nvidia-smi), **AMD** (via rocm-smi/sysfs), **Intel** (via lspci), and **Apple** (via system_profiler or sysfs) GPUs.
- Memory bandwidth resolution combines a curated `GPU_BANDWIDTH` table with the **`dbgpu`** database for comprehensive coverage.
- All detectors implement **fail-safe error handling**, returning empty lists on failure to ensure the scan completes regardless of individual tool availability.
- The architecture handles edge cases like **unified memory** GPUs (GB10, DGX SPARK) by substituting system RAM when dedicated VRAM is unavailable.

## Frequently Asked Questions

### How does WhichLLM handle missing GPU drivers or tools?

When specific tools like `nvidia-smi` or `rocm-smi` are missing, WhichLLM automatically falls back to alternative detection methods such as parsing `/sys/class/drm` entries or using WMI queries. If all methods fail, the detector catches exceptions and returns an empty list, allowing the overall hardware scan to continue with other vendors.

### Can WhichLLM detect Apple Silicon GPUs on Linux?

Yes, WhichLLM detects Apple Silicon GPUs running under Linux via the `detect_apple_gpu_linux()` function, which reads DRM sysfs entries in `/sys/class/drm`. This supports Asahi Linux installations and other Apple-on-Linux configurations that expose hardware information through the standard DRM interface.

### What information does WhichLLM extract from NVIDIA GPUs?

Beyond basic name and VRAM, the `detect_nvidia_gpus()` function extracts CUDA driver versions, compute capability ratings (via `NVIDIA_COMPUTE_CAPABILITY`), and memory bandwidth (via `resolve_detected_bandwidth`). For unified-memory architectures like the DGX SPARK, it also identifies when GPUs share system memory.

### Is the hardware detection cross-platform?

Yes, the detection system supports Linux, macOS, and Windows through platform-specific implementations. The orchestrator uses `platform.system()` to determine the host OS and dispatches to appropriate detectors, ensuring that WhichLLM provides accurate hardware profiles regardless of whether you are running on Ubuntu, macOS Sonoma, or Windows 11.