# Which Libraries Does WhichLLM Use for NVIDIA GPU Detection?

> Discover which libraries WhichLLM uses for NVIDIA GPU detection. Learn how pynvml and nvidia-smi enable efficient GPU monitoring for your LLM projects.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: internals
- Published: 2026-06-10

---

**WhichLLM relies on the `pynvml` library as its primary method for NVIDIA GPU detection, with a robust fallback to the `nvidia-smi` command-line utility invoked via Python's `subprocess` module when NVML initialization fails.**

WhichLLM is an open-source tool designed to match language models with appropriate hardware configurations. To accurately inventory NVIDIA GPUs, the project implements a dual-path detection system in [`src/whichllm/hardware/nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/nvidia.py) that combines direct library integration with system command execution. The libraries used for NVIDIA GPU detection in WhichLLM are specifically chosen to ensure reliable hardware enumeration while maintaining graceful degradation when Python dependencies are unavailable.

## Primary Detection via pynvml

The primary detection path leverages **`pynvml`**, a Python wrapper for the NVIDIA Management Library (NVML). In [`src/whichllm/hardware/nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/nvidia.py), the module attempts to import `pynvml` at line 10 and initializes the NVML session via `nvmlInit()` to query GPU properties directly from the driver.

When available, WhichLLM calls several NVML functions to populate hardware metadata:

- `nvmlDeviceGetCount()` – enumerates available GPUs
- `nvmlDeviceGetName()` – retrieves the GPU model identifier
- `nvmlDeviceGetMemoryInfo()` – obtains VRAM capacity in bytes
- `nvmlSystemGetDriverVersion()` – extracts the CUDA driver version

These data points are used to instantiate `GPUInfo` objects defined in [`src/whichllm/hardware/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/types.py). The implementation handles specialized cases such as unified-memory GPUs (Apple Silicon or integrated graphics) and extracts compute capability information where available.

### NVML Implementation Details

The core logic resides in lines 122-148 of [`nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/nvidia.py), where the code iterates through detected devices and constructs hardware profiles. The implementation wraps NVML calls in `try/except` blocks (lines 112-119) to catch initialization failures, immediately triggering the fallback mechanism if `pynvml` cannot interface with the NVIDIA driver.

## Fallback Detection via nvidia-smi

When `pynvml` is not installed or NVML initialization fails, WhichLLM automatically switches to a **subprocess-based approach** using the system's `nvidia-smi` binary. This fallback is implemented in the `_detect_nvidia_gpus_via_smi` function (lines 71-80 of [`nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/nvidia.py)).

The fallback mechanism executes `nvidia-smi` with specific CSV formatting flags via `subprocess.run()`, then parses the output using regular expressions to extract GPU names and memory statistics. This approach requires no Python dependencies beyond the standard library but depends on the NVIDIA driver utilities being installed and available in the system PATH.

### Error Handling Strategy

The detection logic uses guarded import statements and conditional initialization:

```python

# src/whichllm/hardware/nvidia.py (lines 10-14)

try:
    import pynvml
    NVML_AVAILABLE = True
except ImportError:
    NVML_AVAILABLE = False

```

This pattern ensures that WhichLLM remains functional even in environments without NVIDIA hardware or the NVML Python bindings, allowing the tool to proceed with CPU-only model recommendations.

## Implementation Architecture

The NVIDIA detection module follows a layered architecture:

| Component | File Path | Responsibility |
|-----------|-----------|----------------|
| Detection Engine | [`src/whichllm/hardware/nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/nvidia.py) | Implements both NVML and nvidia-smi detection paths |
| Data Models | [`src/whichllm/hardware/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/types.py) | Defines `GPUInfo` dataclass for hardware metadata |
| Test Suite | [`tests/test_nvidia_detection.py`](https://github.com/Andyyyy64/whichllm/blob/main/tests/test_nvidia_detection.py) | Validates both primary and fallback detection paths |

The `detect_nvidia_gpus()` function serves as the public API, automatically selecting the appropriate backend based on environment availability.

## Practical Usage Example

To detect NVIDIA GPUs in a Python environment using WhichLLM:

```python
from whichllm.hardware.nvidia import detect_nvidia_gpus

# Detect NVIDIA GPUs on the current machine

gpus = detect_nvidia_gpus()

for gpu in gpus:
    print(f"Name: {gpu.name}")
    print(f"VRAM: {gpu.vram_bytes / (1024**3):.1f} GiB")
    print(f"CUDA version: {gpu.cuda_version}")
    print(f"Compute capability: {gpu.compute_capability}")
    print("-" * 30)

```

If `pynvml` is unavailable, the same function automatically switches to the `nvidia-smi` fallback without requiring any code changes or additional configuration.

## Summary

- **Primary library**: `pynvml` provides direct NVML access for GPU enumeration, VRAM detection, and CUDA version queries in [`src/whichllm/hardware/nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/nvidia.py)
- **Fallback mechanism**: Python's built-in `subprocess` module executes `nvidia-smi` when NVML is unavailable (lines 71-80)
- **Automatic selection**: The `detect_nvidia_gpus()` function handles backend selection transparently via `try/except` blocks (lines 10-14 and 112-119)
- **Data modeling**: Hardware metadata is standardized through the `GPUInfo` class defined in [`src/whichllm/hardware/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/types.py)

## Frequently Asked Questions

### What is the primary library for NVIDIA GPU detection in WhichLLM?

The primary library is **`pynvml`**, a Python wrapper for the NVIDIA Management Library (NVML). It provides direct access to GPU properties including device names, memory capacity, and driver versions through the NVIDIA driver interface.

### How does WhichLLM handle environments where pynvml is not installed?

WhichLLM implements a **graceful fallback** using Python's `subprocess` module to execute the `nvidia-smi` command-line utility. This occurs automatically in the `_detect_nvidia_gpus_via_smi` function when the `pynvml` import fails or NVML initialization raises an exception.

### Can WhichLLM detect NVIDIA GPUs without NVIDIA drivers installed?

**No.** Both detection methods require NVIDIA drivers to be present on the system. The `pynvml` library requires the NVML shared libraries included with the driver package, while the `nvidia-smi` fallback relies on the command-line utility that ships with NVIDIA drivers. Without drivers, neither path can enumerate hardware.

### Where is the GPU detection logic implemented in the WhichLLM codebase?

The core detection logic resides in **[`src/whichllm/hardware/nvidia.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/nvidia.py)**, specifically in the `detect_nvidia_gpus()` function (primary NVML path, lines 122-148) and the `_detect_nvidia_gpus_via_smi()` helper (fallback path, lines 71-80). The data structures used to represent GPU information are defined in [`src/whichllm/hardware/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/types.py).