How llmfit Detects Hardware Specifications: RAM, CPU, and GPU Detection Explained

llmfit detects hardware specifications by calling SystemSpecs::detect() in llmfit-core/src/hardware.rs, which orchestrates platform-specific probes using the sysinfo crate for CPU/RAM, vendor-specific CLI tools like nvidia-smi and rocm-smi for GPUs, and OS APIs such as Windows WMI and macOS system_profiler to build a complete system profile.

llmfit is an open-source Rust project that automatically profiles your machine to determine model compatibility and inference backends. Understanding how llmfit detects hardware specifications reveals a layered detection strategy that handles Linux, macOS, and Windows while accommodating everything from consumer GPUs to unified-memory SoCs like Apple Silicon and NVIDIA Grace.

The Entry Point: SystemSpecs::detect()

All hardware detection flows through the SystemSpecs::detect() method in [hardware.rs](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs). This function implements a layered detection pipeline that first collects basic system information, then iterates through vendor-specific GPU probes, and finally aggregates the results into a portable struct.

The detection sequence follows a strict order: CPU and RAM collection occurs first (lines 73‑90), followed by GPU enumeration via detect_all_gpus() (lines 98‑115), and concludes with backend inference selection based on the discovered hardware.

Detecting CPU and System Memory

Cross-Platform CPU and RAM Collection

For CPU cores, total RAM, and available RAM, llmfit relies on the sysinfo crate. The implementation calls System::new_all() to refresh hardware information, then extracts:

• sys.total_memory() and sys.available_memory() for memory statistics
• sys.cpus() to count physical cores and retrieve the CPU name

These values are converted from bytes to gigabytes and stored in the SystemSpecs struct. This approach works uniformly across Linux, macOS, and Windows without platform-specific code for the initial collection.

Windows-Specific AMD APU Corrections

On Windows systems with AMD APUs that reserve system RAM for GPU unified memory architecture (UMA), the sysinfo crate reports incorrect total memory. To fix this, llmfit calls windows_apu_total_ram_gb() (lines 1192‑1199), which queries Win32_PhysicalMemory via WMI to obtain the actual DIMM total. When a memory carve-out is detected, this value overrides the standard sysinfo measurement, ensuring accurate RAM calculations for APU-based systems.

GPU Detection Architecture

The detect_all_gpus Orchestrator

GPU detection centers on detect_all_gpus(total_ram_gb, &cpu_name) (lines 98‑115). This function aggregates results from multiple vendor-specific probes into a Vec<GpuInfo>, sorted by available VRAM. The orchestrator handles:

• Vendor CLI tools (nvidia-smi, rocm-smi, npu-smi)
• OS-specific APIs (Windows WMI, macOS Metal)
• Linux sysfs fallbacks when CLI tools are unavailable
• Vulkan as a generic fallback for unidentified GPUs

After collection, the function merges duplicate entries, counts identical GPUs, and selects the primary GPU (the one with the most VRAM) for backend selection.

NVIDIA GPU Detection via nvidia-smi

For NVIDIA GPUs, llmfit attempts two query strategies in detect_nvidia_gpus() (lines 165‑176):

1. Extended query: Runs nvidia-smi --query-gpu=addressing_mode,memory.total,name to detect unified memory support, parsed by parse_nvidia_smi_extended() (lines 49‑66)
2. Standard fallback: Uses nvidia-smi --query-gpu=memory.total,name parsed by parse_nvidia_smi_list()

If nvidia-smi is unavailable (common in containerized environments), llmfit falls back to reading Linux sysfs entries under /sys/class/drm via detect_nvidia_gpu_sysfs_info() (lines 96‑126) to extract VRAM and device information.

AMD GPU Detection via rocm-smi and sysfs

AMD GPU detection follows a similar dual-path approach in detect_amd_gpu_rocm_info() (lines 78‑85):

• ROCm path: Executes rocm-smi --showmeminfo vram and rocm-smi --showproductname, then parses the output to extract per-card VRAM and model names
• Sysfs fallback: If ROCm is unavailable, detect_amd_gpu_sysfs_info() (lines 166‑186) reads /sys/class/drm for VRAM and calls lspci to resolve human-readable GPU names

Intel and Apple Silicon GPU Detection

Intel GPUs are handled by detect_intel_gpus() (lines 50‑52), which uses lspci on Linux and Metal frameworks on macOS to identify integrated and discrete Intel graphics.

For Apple Silicon and macOS discrete GPUs, detect_macos_metal_gpus() (lines 62‑68) invokes system_profiler SPDisplaysDataType to extract GPU names and VRAM. Apple Silicon detection receives special handling in detect_apple_gpu() (lines 71‑85), which correctly identifies unified memory architectures and marks the system as having shared RAM/VRAM.

Ascend NPUs and Vulkan Fallback

llmfit supports Huawei Ascend NPUs through detect_ascend_npus() (line 86), which executes npu-smi to retrieve NPU names and memory statistics.

When no vendor-specific tool succeeds, the system calls detect_vulkan_gpu_info() and merges results via merge_gpu_sources(), providing generic GPU information for unrecognized hardware.

Handling Unified Memory and Multi-GPU Setups

After collecting all GPUs, detect_all_gpus() checks for unified memory systems using is_nvidia_unified_memory_gpu() (lines 34‑42). When detected (as with NVIDIA Grace, DGX Spark, or Apple Silicon), the code overwrites per-GPU VRAM with total_ram_gb, reflecting the shared physical memory pool.

For multi-GPU configurations, llmfit groups identical GPU models and tracks a count field in GpuInfo. This allows accurate calculation of total_gpu_vram_gb by summing per-card VRAM multiplied by the card count, essential for tensor-parallel inference planning.

Backend Selection Logic

The final detection phase determines which backend llmfit will use for model fitting. If a GPU is present, the backend is derived from the primary GPU's vendor (e.g., Cuda, Metal, Rocm). If no GPU is detected, llmfit falls back to CPU-specific backends:

• CpuArm for aarch64 architectures (Apple Silicon, ARM servers)
• CpuX86 for x86_64 systems

This logic resides in the backend selection phase (lines 122‑128) of the detection pipeline.

The SystemSpecs Output Structure

The detection process populates a comprehensive SystemSpecs struct:

pub struct SystemSpecs {
    pub total_ram_gb: f64,
    pub available_ram_gb: f64,
    pub total_cpu_cores: usize,
    pub cpu_name: String,
    pub has_gpu: bool,
    pub gpu_vram_gb: Option<f64>,
    pub total_gpu_vram_gb: Option<f64>,
    pub gpu_available_gb: Option<f64>,
    pub gpu_name: Option<String>,
    pub gpu_count: u32,
    pub unified_memory: bool,
    pub backend: GpuBackend,
    pub gpus: Vec<GpuInfo>,
    pub cluster_mode: bool,
    pub cluster_node_count: u32,
}

Practical Usage Example

To access hardware specifications in your own Rust code using llmfit:

use llmfit_core::hardware::SystemSpecs;

// Get a snapshot of the current machine.
let specs = SystemSpecs::detect();

println!("CPU: {} ({} cores)", specs.cpu_name, specs.total_cpu_cores);
println!("RAM: {:.1} GB total, {:.1} GB available", specs.total_ram_gb, specs.available_ram_gb);

if specs.has_gpu {
    println!("Primary GPU: {} ({} GB VRAM)", 
        specs.gpu_name.unwrap_or_else(|| "Unnamed".into()),
        specs.gpu_vram_gb.map_or(0.0, |v| v));
    println!("All GPUs: {:?}", specs.gpus);
    println!("Backend used for inference: {}", specs.backend.label());
} else {
    println!("No GPU detected – falling back to CPU ({})", specs.backend.label());
}

Running this snippet produces a concise hardware summary that feeds into llmfit's model-selection algorithms.

Summary

• Central detection occurs in SystemSpecs::detect() within llmfit-core/src/hardware.rs, implementing a layered, cross-platform probe strategy.
• CPU and RAM detection uses the sysinfo crate for basic metrics, with Windows-specific WMI corrections for AMD APUs that reserve system memory.
• GPU discovery orchestrates vendor-specific tools: nvidia-smi for NVIDIA, rocm-smi for AMD, system_profiler for macOS Metal, and lspci/sysfs for fallback scenarios.
• Unified memory systems (Apple Silicon, NVIDIA Grace) are detected via addressing-mode queries and flag-setting, with total system RAM reported as VRAM.
• Multi-GPU support aggregates identical cards and calculates total VRAM across all detected devices.
• Backend inference automatically selects Cuda, Metal, Rocm, Vulkan, Sycl, CpuArm, or CpuX86 based on detected hardware.

Frequently Asked Questions

How does llmfit handle unified memory systems like Apple Silicon?

llmfit detects unified memory by checking GPU addressing modes (via nvidia-smi on NVIDIA Grace) or platform detection (via system_profiler on macOS). When unified memory is confirmed in detect_all_gpus() (lines 34‑42), the system sets unified_memory: true and overwrites the GPU's VRAM value with total_ram_gb, ensuring model fit calculations account for the shared memory pool.

What happens if nvidia-smi is not available on a Linux system?

If nvidia-smi is absent or fails, llmfit falls back to detect_nvidia_gpu_sysfs_info() (lines 96‑126), which reads device information directly from /sys/class/drm. This method extracts VRAM and GPU identifiers from the kernel's DRM subsystem, allowing detection even in minimal container environments without NVIDIA utilities installed.

How does llmfit distinguish between multiple identical GPUs?

The detect_all_gpus() function groups GPUs by model name and VRAM, storing a count field in the GpuInfo struct for identical cards. Rather than duplicating entries, llmfit tracks the quantity of each GPU model and calculates total_gpu_vram_gb by multiplying per-card VRAM by the card count, supporting accurate capacity planning for multi-GPU inference.

Does llmfit support hardware detection on Windows?

Yes, llmfit fully supports Windows through multiple mechanisms: it uses the sysinfo crate for basic CPU/RAM data, Windows WMI (Win32_PhysicalMemory) to correct AMD APU memory reporting in windows_apu_total_ram_gb() (lines 1192‑1199), and nvidia-smi PowerShell integration for NVIDIA GPU detection. The detection pipeline abstracts platform differences, offering the same SystemSpecs output on Windows, Linux, and macOS.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}