How WhichLLM Detects GPU Hardware: Cross-Platform GPU Detection Explained

WhichLLM detects GPU hardware using a modular, OS-aware orchestration system that aggregates vendor-specific detectors into a unified HardwareInfo object, ensuring robust cross-platform compatibility across Linux, macOS, and Windows.

The whichllm repository implements a sophisticated hardware detection layer designed to identify GPUs from NVIDIA, AMD, Intel, and Apple across different operating systems. By combining native system tools, Python bindings, and hardware databases, the detection system provides detailed GPU specifications including VRAM, compute capability, and memory bandwidth. This architecture ensures that WhichLLM can recommend optimal large language models based on your actual hardware capabilities.

The Detection Orchestrator

At the heart of WhichLLM's GPU detection lies the detect_hardware() function in src/whichllm/hardware/detector.py. This central orchestrator determines the host operating system using platform.system() and dispatches to appropriate vendor-specific detectors based on the platform.

OS Detection and Dispatch Logic

The orchestrator implements a prioritized detection strategy:

• NVIDIA – Always queried regardless of OS via detect_nvidia_gpus()
• Linux – Checks AMD (detect_amd_gpus), Intel (detect_intel_gpus), and Apple-on-Linux (detect_apple_gpu_linux)
• macOS – Queries Apple silicon via detect_apple_gpu()
• Windows – Uses generic Windows GPU detection (detect_windows_gpus) for non-NVIDIA hardware

Each detector operates independently and returns a list of GPU objects, which the orchestrator aggregates into the final HardwareInfo result.

Vendor-Specific Detection Strategies

NVIDIA GPU Detection (NVML and nvidia-smi)

The detect_nvidia_gpus() function in src/whichllm/hardware/nvidia.py implements a two-tier fallback system. It first attempts to use the NVML Python bindings (pynvml) for direct hardware communication. If NVML initialization fails or the library is unavailable, it falls back to parsing the nvidia-smi CLI output.

For each detected device, the module collects:

• GPU name and VRAM capacity
• CUDA driver version
• Compute capability (via NVIDIA_COMPUTE_CAPABILITY lookup)
• Memory bandwidth (via resolve_detected_bandwidth)

The system also handles unified-memory GPUs such as the "GB10" and "DGX SPARK" by treating system RAM as VRAM when dedicated video memory is not present.

AMD GPU Detection (rocm-smi and sysfs)

AMD detection in src/whichllm/hardware/amd.py follows a similar dual-path approach. The primary method executes rocm-smi with JSON output formatting to extract GPU specifications. When rocm-smi is unavailable, the detector falls back to reading sysfs DRM entries in /sys/class/drm to obtain vendor IDs, device IDs, and VRAM information directly from the kernel.

Intel Integrated GPU Detection (lspci)

For Intel graphics, the detect_intel_gpus() function in src/whichllm/hardware/intel.py executes lspci -nn -k and parses the output for Intel graphics devices. It extracts VRAM specifications where reported by the hardware, making it compatible with both integrated and discrete Intel GPUs on Linux systems.

Apple GPU Detection (macOS and Linux)

Apple hardware detection splits across two functions in src/whichllm/hardware/apple.py:

• macOS: detect_apple_gpu() parses output from system_profiler SPDisplaysDataType to obtain GPU names and memory configurations
• Linux: detect_apple_gpu_linux() handles Asahi driver installations by reading /sys/class/drm entries, similar to the AMD sysfs fallback method

Windows Non-NVIDIA Detection (WMI)

The detect_windows_gpus() function in src/whichllm/hardware/windows.py utilizes WMI (win32com.client) to query the Win32_VideoController class. This extracts GPU names, RAM amounts, driver versions, and vendor information for non-NVIDIA hardware on Windows systems. NVIDIA detection on Windows is delegated to the dedicated NVIDIA module to avoid duplicate entries.

Memory Bandwidth Resolution

After detecting GPU hardware, WhichLLM resolves memory bandwidth specifications through the resolve_detected_bandwidth() function in src/whichllm/hardware/gpu_db.py. This function first checks a curated GPU_BANDWIDTH lookup table for known hardware configurations. If the GPU is not found in the local table, it falls back to querying the dbgpu database for comprehensive coverage of modern and legacy graphics cards.

Fail-Safe Architecture

All detector functions in WhichLLM implement fail-safe error handling. Each detector wraps its execution in exception handlers that catch any errors, log the failure, and return an empty list rather than raising exceptions. This design ensures that a failure to detect one GPU vendor never aborts the entire hardware scan, allowing WhichLLM to provide partial hardware information even when specific drivers or tools are missing.

Practical Usage Examples

To run the complete hardware detection stack in your Python application:

from whichllm.hardware.detector import detect_hardware

# Run the orchestrated detection

info = detect_hardware()

# Print a summary of detected GPUs

for gpu in info.gpus:
    print(f"GPU: {gpu.name}")
    print(f"  Vendor: {gpu.vendor}")
    print(f"  VRAM: {gpu.vram_bytes // (1024**3)} GiB")
    print(f"  Compute Capability: {gpu.compute_capability}")
    print(f"  Memory Bandwidth: {gpu.memory_bandwidth_gbps} GB/s")
    print(f"  Unified Memory: {gpu.shared_memory}")
    print()

For targeted NVIDIA detection without running the full orchestrator:

from whichllm.hardware.nvidia import detect_nvidia_gpus

nvidia_gpus = detect_nvidia_gpus()
if nvidia_gpus:
    print("Detected NVIDIA GPUs:", [g.name for g in nvidia_gpus])
else:
    print("No NVIDIA GPUs found.")

Summary

• WhichLLM detects GPU hardware through a centralized orchestrator in detector.py that dispatches OS-specific detection routines.
• The system supports NVIDIA (via NVML/nvidia-smi), AMD (via rocm-smi/sysfs), Intel (via lspci), and Apple (via system_profiler or sysfs) GPUs.
• Memory bandwidth resolution combines a curated GPU_BANDWIDTH table with the dbgpu database for comprehensive coverage.
• All detectors implement fail-safe error handling, returning empty lists on failure to ensure the scan completes regardless of individual tool availability.
• The architecture handles edge cases like unified memory GPUs (GB10, DGX SPARK) by substituting system RAM when dedicated VRAM is unavailable.

Frequently Asked Questions

How does WhichLLM handle missing GPU drivers or tools?

When specific tools like nvidia-smi or rocm-smi are missing, WhichLLM automatically falls back to alternative detection methods such as parsing /sys/class/drm entries or using WMI queries. If all methods fail, the detector catches exceptions and returns an empty list, allowing the overall hardware scan to continue with other vendors.

Can WhichLLM detect Apple Silicon GPUs on Linux?

Yes, WhichLLM detects Apple Silicon GPUs running under Linux via the detect_apple_gpu_linux() function, which reads DRM sysfs entries in /sys/class/drm. This supports Asahi Linux installations and other Apple-on-Linux configurations that expose hardware information through the standard DRM interface.

What information does WhichLLM extract from NVIDIA GPUs?

Beyond basic name and VRAM, the detect_nvidia_gpus() function extracts CUDA driver versions, compute capability ratings (via NVIDIA_COMPUTE_CAPABILITY), and memory bandwidth (via resolve_detected_bandwidth). For unified-memory architectures like the DGX SPARK, it also identifies when GPUs share system memory.

Is the hardware detection cross-platform?

Yes, the detection system supports Linux, macOS, and Windows through platform-specific implementations. The orchestrator uses platform.system() to determine the host OS and dispatches to appropriate detectors, ensuring that WhichLLM provides accurate hardware profiles regardless of whether you are running on Ubuntu, macOS Sonoma, or Windows 11.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}