What Hardware Information Does llmfit Collect? A Complete Technical Guide

llmfit collects comprehensive hardware telemetry—including total and available system RAM, CPU core count and model name, and detailed GPU metrics such as VRAM capacity, device count, backend type (CUDA/Metal/ROCm), and unified memory status—via the SystemSpecs::detect() method in llmfit-core/src/hardware.rs.

The llmfit CLI and library automatically probe your machine's compute resources to determine optimal model placement and inference backends. Understanding exactly what hardware information llmfit collects helps developers interpret compatibility reports and debug deployment issues across diverse environments from single GPUs to multi-node DGX clusters.

Core Hardware Detection Entry Point

All hardware discovery originates from the SystemSpecs::detect() function located in llmfit-core/src/hardware.rs (lines 42-70). This method orchestrates platform-specific probes to populate the SystemSpecs struct, which serves as the canonical representation of your machine's capabilities. The detection flow first queries general system resources, then executes vendor-specific GPU discovery via detect_all_gpus (line 59), and finally aggregates results to identify the primary compute device.

System Memory and CPU Metrics

RAM Statistics

llmfit gathers precise memory metrics to calculate available workspace for model weights and context windows. The system collects:

• total_ram_gb: Total installed system RAM
• available_ram_gb: Currently unallocated RAM available for inference

These values derive from the sysinfo crate's total_memory() and available_memory() methods, converted from bytes to gigabytes (lines 73-85). This data enables llmfit to prevent out-of-memory errors when loading large models.

CPU Detection and Architecture Classification

The library captures processor details to select appropriate optimized backends:

• total_cpu_cores: Logical core count via sys.cpus().len() (line 88)
• cpu_name: Human-readable processor identifier via Self::detect_cpu_name(&sys) (line 89)
• Implicit backend selection: ARM architectures map to CpuArm, while x86 systems use CpuX86

These metrics determine whether llmfit selects NEON (ARM) or AVX2/AVX-512 (x86) vectorized inference paths.

GPU Detection and Aggregation

Primary GPU Metrics

When discrete accelerators are present, llmfit identifies the optimal primary device by sorting detected GPUs by VRAM descending (lines 100-115). The system records:

• has_gpu: Boolean presence flag
• gpu_name: Primary device name (e.g., "NVIDIA GeForce RTX 4090")
• gpu_vram_gb: VRAM capacity of the primary GPU (the card with most memory)
• backend: Acceleration API enum (GpuBackend) indicating CUDA, Metal, ROCm, Vulkan, SYCL, or other providers (defined in llmfit-core/src/providers.rs)

Multi-GPU and Cluster Support

For distributed inference environments, llmfit aggregates resources across all identical GPUs:

• total_gpu_vram_gb: Combined VRAM across all same-model GPUs
• gpu_count: Number of identical accelerator cards
• cluster_mode: Boolean flag for multi-node setups
• cluster_node_count: Node count in cluster configurations (lines 66-69)

This aggregation supports DGX-Spark and other multi-node inference scenarios where model parallelism requires total pool memory calculations.

Per-GPU Detailed Records

Individual device specifications hash stored in the gpus: Vec<GpuInfo> field. The GpuInfo struct (lines 32-40) captures:

• name: Device identifier string
• vram_gb: Optional memory capacity (may be estimated on certain platforms)
• backend: Specific acceleration API for this device
• count: Quantity of identical cards
• unified_memory: Boolean indicating shared system/GPU memory architecture

Unified Memory Detection

llmfit specifically detects unified memory architectures common in Apple Silicon, AMD APUs, and NVIDIA Grace Hopper systems:

• unified_memory: True when system RAM is shared with the GPU
• gpu_available_gb: On Apple Silicon, queries Metal's recommendedMaxWorkingSetSize to determine the GPU's actual allocatable portion of the unified pool (lines 130-138)

This distinction prevents overallocation on systems where the GPU competes with the OS for RAM.

Platform-Specific Detection Methods

The detect_all_gpus implementation employs a cascading fallback strategy across operating systems:

Probe Method	Platforms	Tools/APIs
NVIDIA GPUs	Linux, Windows, macOS	nvidia-smi (standard and extended queries), sysfs (/sys/class/drm), lspci, Windows WMI
AMD GPUs	Linux, Windows, macOS	rocm-smi, sysfs enumeration, lspci, macOS system_profiler
Intel GPUs	Linux, Windows, macOS	lspci, Windows WMI, macOS Metal
Apple Silicon	macOS (ARM64)	Metal API (recommendedMaxWorkingSetSize)
NVIDIA Grace/DGX	Linux	ATS addressing mode detection via nvidia-smi
Ascend NPUs	Linux	npu-smi
Vulkan Fallback	Android/Termux, Generic	Vulkan driver queries
Windows Registry	Windows	HKLM registry reads for exact VRAM byte values

The detection logic filters out integrated GPUs when discrete cards are present, ensuring llmfit prioritizes high-performance accelerators for inference workloads.

Programmatic Hardware Inspection

Access hardware telemetry directly in Rust applications using the llmfit_core crate:

use llmfit_core::hardware::SystemSpecs;

fn main() {
    // Detect the current machine’s hardware
    let specs = SystemSpecs::detect();

    println!("🖥️  CPU: {} ({} cores)", specs.cpu_name, specs.total_cpu_cores);
    println!("💾  RAM: {:.2} GB total, {:.2} GB available", 
             specs.total_ram_gb, specs.available_ram_gb);

    if specs.has_gpu {
        println!("🖧  GPU: {}", specs.gpu_name.unwrap_or_else(|| "unknown".into()));
        println!("   VRAM (primary): {:.2} GB", specs.gpu_vram_gb.unwrap_or(0.0));
        println!("   VRAM (total across GPUs): {:.2} GB", 
                 specs.total_gpu_vram_gb.unwrap_or(0.0));
        println!("   GPU count: {}", specs.gpu_count);
        println!("   Backend: {}", specs.backend.label());
        
        if specs.unified_memory {
            println!("   Unified memory (system RAM shared with GPU)");
            if let Some(avail) = specs.gpu_available_gb {
                println!("   GPU-available unified memory: {:.2} GB", avail);
            }
        }
    } else {
        println!("⚡  No GPU detected – CPU‑only inference will be used.");
    }
}

This programmatic interface enables custom tooling and automated benchmarking pipelines to leverage llmfit's detection logic.

CLI Hardware Inspection

The system subcommand provides immediate terminal visibility into detected resources:

$ llmfit system
CPU: Intel(R) Core(TM) i7-12700K (12 cores)
RAM: 32.00 GB total, 28.70 GB available
GPU: NVIDIA GeForce RTX 4090
VRAM (primary): 24.00 GB
VRAM (total across GPUs): 48.00 GB
GPU count: 2
Backend: CUDA

The CLI implementation resides in llmfit-tui/src/main.rs, invoking the same SystemSpecs::detect() function used by the analysis engine.

Summary

• llmfit probes memory, CPU, and GPU subsystems through SystemSpecs::detect() in llmfit-core/src/hardware.rs to build a complete compute profile.
• GPU detection cascades through vendor tools (nvidia-smi, rocm-smi, Metal) before falling back to generic PCI enumeration, capturing VRAM, device count, and backend APIs.
• Unified memory architectures (Apple Silicon, NVIDIA Grace) receive specialized handling to query available GPU memory within shared pools.
• Multi-GPU and cluster configurations are supported through aggregate VRAM calculations and node counting fields.
• Both programmatic Rust APIs and CLI interfaces expose this hardware information for integration into deployment workflows.

Frequently Asked Questions

Does llmfit support multi-GPU inference setups?

Yes. llmfit detects all identical GPUs and aggregates their VRAM into total_gpu_vram_gb while maintaining individual device records in the gpus vector. The system populates gpu_count to indicate how many accelerators are available for model parallelism, and supports DGX-Spark style clusters through cluster_mode and cluster_node_count fields (lines 66-69).

How does llmfit detect unified memory on Apple Silicon?

On macOS ARM64 systems, llmfit sets the unified_memory flag to true and queries Metal's recommendedMaxWorkingSetSize API to populate gpu_available_gb (lines 130-138). This returns the actual amount of the unified pool the GPU can wire, rather than reporting total system RAM, preventing context length calculations from oversubscribing memory shared with the operating system.

Can I use llmfit's hardware detection in my own Rust project?

Absolutely. Import the llmfit_core crate and call SystemSpecs::detect() to receive a fully populated struct containing all hardware metrics. The detection logic in llmfit-core/src/hardware.rs has no external dependencies on the llmfit CLI, making it suitable for integration into custom MLOps tooling or benchmark suites.

What happens if no GPU is detected?

When detect_all_gpus finds no accelerators, has_gpu returns false and llmfit defaults to CPU-only inference pathways. The system still reports RAM and CPU metrics via total_ram_gb, available_ram_gb, and total_cpu_cores, allowing the analysis engine in llmfit-core/src/analysis.rs to recommend quantized models that fit within system memory constraints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}