GPU Detection Methods in llmfit: How It Identifies NVIDIA, AMD, and Apple Silicon

llmfit detects NVIDIA GPUs via nvidia-smi, AMD GPUs via rocm-smi, and Apple Silicon Macs via system_profiler, parsing command output into internal GpuInfo structures within the hardware.rs module.

The llmfit repository implements cross-platform GPU detection to optimize model fitting and inference performance across diverse hardware configurations. The detection mechanism relies on platform-specific command-line utilities that query system specifications and populate internal data structures used throughout the analysis pipeline.

NVIDIA GPU Detection via nvidia-smi

For NVIDIA hardware, llmfit executes the nvidia-smi command—the standard NVIDIA System Management Interface utility. The implementation in llmfit-core/src/hardware.rs invokes this command and parses its stdout to extract critical GPU metadata.

The detection routine captures:

• GPU model name
• Driver version
• Total available VRAM

This information is converted into the internal GpuInfo structure, which represents the detected hardware capabilities. The SystemSpecs::detect() function handles the command execution and output parsing, ensuring compatibility with consumer and datacenter NVIDIA GPUs.

AMD GPU Detection via rocm-smi

On AMD platforms, llmfit utilizes the rocm-smi utility—AMD's ROCm System Management Interface. This command provides equivalent functionality to nvidia-smi for Radeon and Instinct hardware.

The detection process reads:

• GPU name and architecture
• Memory capacity (HBM or GDDR)
• Driver stack version

As implemented in the llmfit-core crate, the AMD detection path follows the same pattern as NVIDIA: command invocation, stdout parsing, and conversion to the standardized GpuInfo representation used by downstream components.

Apple Silicon Detection via system_profiler

For Apple Silicon machines (M1, M2, M3 series), llmfit employs system_profiler SPDisplaysDataType to query the graphics subsystem. This macOS system utility reports integrated GPU specifications without requiring third-party drivers.

Because Apple Silicon utilizes unified memory architecture—where system RAM serves as both CPU memory and GPU VRAM—the detection logic treats the reported memory capacity as available VRAM for inference calculations. This distinction is documented in the repository's AGENTS.md under the Platform notes section, which explains how unified memory affects the min_vram_gb calculations used in models.rs.

Implementation Architecture in hardware.rs

The core detection logic resides in llmfit-core/src/hardware.rs, where the SystemSpecs::detect() method orchestrates platform identification:

1. Detects the operating system and available utilities
2. Executes the appropriate command (nvidia-smi, rocm-smi, or system_profiler)
3. Parses command output into structured GpuInfo instances
4. Populates the SystemSpecs struct with available GPU resources

These GPU specifications propagate to llmfit-core/src/models.rs, which calculates min_vram_gb requirements for different model configurations. The fit.rs module then applies this hardware information when scoring model fit and selecting optimal execution modes.

Practical Usage Examples

CLI Detection Output

When running llmfit on a system with NVIDIA hardware, the tool automatically identifies the GPU:

cargo run -- --cli

# Output includes:

#   GPU: NVIDIA GeForce RTX 3080 (VRAM: 10 GB)

On Apple Silicon, the unified memory architecture is explicitly noted:

cargo run -- --cli

# Output shows:

#   GPU: Apple M1 Pro (Unified Memory: 16 GB)

Programmatic GPU Access

Developers can access detected GPU information programmatically using the SystemSpecs API:

use llmfit_core::hardware::SystemSpecs;

fn main() {
    // Detect system specs, including GPU
    let specs = SystemSpecs::detect().expect("Failed to detect hardware");
    if let Some(gpu) = specs.gpus.first() {
        println!("Detected GPU: {} – VRAM: {} GB", gpu.name, gpu.vram_gb);
    }
}

Summary

• NVIDIA detection uses the nvidia-smi command to extract GPU model, driver, and VRAM information.
• AMD detection relies on rocm-smi for ROCm-compatible hardware profiling.
• Apple Silicon detection calls system_profiler SPDisplaysDataType, treating unified memory as available GPU VRAM.
• The implementation lives in llmfit-core/src/hardware.rs, specifically within the SystemSpecs::detect() function.
• Detected specifications inform VRAM requirements in models.rs and execution mode selection in fit.rs.

Frequently Asked Questions

Which command-line tools does llmfit require for GPU detection?

For NVIDIA GPUs, llmfit requires the NVIDIA driver suite including nvidia-smi. For AMD hardware, it requires the ROCm platform with rocm-smi. Apple Silicon Macs use the built-in system_profiler utility, requiring no additional dependencies. These tools must be available in the system PATH for the detection to succeed.

How does llmfit handle unified memory on Apple Silicon Macs?

On Apple Silicon, llmfit detects total system memory via system_profiler and treats this capacity as available GPU VRAM for inference calculations. Because the M-series chips share memory between the CPU and GPU, the tool does not distinguish between "system RAM" and "dedicated VRAM," unlike the discrete GPU detection paths for NVIDIA and AMD hardware.

Where is the GPU detection logic implemented in the llmfit source code?

The primary implementation resides in llmfit-core/src/hardware.rs, where the SystemSpecs::detect() method executes platform-specific commands and parses their output. The detection strategy is further documented in AGENTS.md under the Platform notes section, while llmfit-core/src/models.rs and fit.rs consume the detected GpuInfo structures.

Can llmfit detect multiple GPUs in a single system?

Yes. The SystemSpecs structure maintains a collection of GpuInfo instances, allowing llmfit to enumerate multiple GPUs. The detection functions in hardware.rs parse command output to identify all available devices, enabling the tool to calculate aggregate VRAM capacity and select appropriate execution strategies for multi-GPU configurations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}