How llmfit Detects Windows APU RAM With BIOS GPU Carveout: A Deep Dive into the WMI-Based Correction

llmfit uses a two-step detection process that re-queries physical DIMM capacity via WMI when it detects an AMD Ryzen AI APU, bypassing the reduced memory figure that Windows reports after BIOS GPU carveout.

When running large language models on Windows laptops with integrated AMD graphics, the operating system often underreports total RAM. This happens because the BIOS reserves a slice of system memory for the GPU's unified memory allocator (UMA carve-out), and Windows only sees what's left. The llmfit project solves this by cross-referencing CPU identity against hardware-level WMI queries to recover the true memory pool.

Why BIOS GPU Carveout Breaks Standard RAM Detection

Modern APUs—accelerated processing units that combine CPU and GPU on one die—use unified memory architecture. The GPU doesn't have dedicated VRAM; instead, it allocates from the same DDR5 DIMMs that feed the CPU.

During boot, the motherboard firmware carves out a fixed region for the GPU. Windows then reports only the remaining capacity to applications. A 32 GB system might appear as 24 GB if 8 GB is reserved for graphics.

Standard Rust crates like sysinfo read what the OS provides. For llmfit, which must determine whether a model fits in available RAM plus VRAM, this underreporting causes false rejections.

Step-by-Step: How llmfit Corrects the Memory Figure

The correction lives in llmfit-core/src/hardware.rs and executes during SystemSpecs::detect().

Step 1: Capture the OS-Reported Baseline

The code first obtains the conservative figure from sysinfo:

let total_ram_bytes = sys.total_memory(); // returns OS-visible RAM only
let total_ram_gb    = total_ram_bytes as f64 / (1024.0 * 1024.0 * 1024.0);

This matches what Task Manager would display—often wrong for APU systems.

Step 2: Detect AMD Ryzen AI and Trigger Correction

Immediately after, detect() branches based on CPU identification:

let total_ram_gb = Self::windows_apu_total_ram_gb(&cpu_name, total_ram_gb);

The helper at hardware.rs#L102-L108 implements the carve-out bypass:

fn windows_apu_total_ram_gb(cpu_name: &str, reported_gb: f64) -> f64 {
    if is_amd_unified_memory_apu(cpu_name) {
        detect_windows_physical_total_ram_gb().unwrap_or(reported_gb)
    } else {
        reported_gb
    }
}

Step 3: Query Physical DIMMs via WMI

When the APU heuristic matches, detect_windows_physical_total_ram_gb() executes a WMI query against Win32_PhysicalMemory. This interface reports raw SPD data from the memory controller, ignoring BIOS reservations:

// Conceptual implementation at hardware.rs#L150-170
let wmi_conn = COMInitialized::new()?;
let results: Vec<Win32_PhysicalMemory> = wmi_conn.query()?;
let total_bytes: u64 = results.iter().map(|m| m.Capacity).sum();
Ok(total_bytes as f64 / (1024.0 * 1024.0 * 1024.0))

The summed Capacity fields reflect the actual DIMM sizes installed—32 GB remains 32 GB regardless of carve-out.

APU GPU Handling: Unifying RAM and VRAM Reports

After memory correction, detect_all_gpus() reconciles the GPU structure. For confirmed AMD unified-memory APUs, it flattens the distinction between system RAM and video memory:

if is_amd_unified_memory_apu(cpu_name) {
    gpus[idx].unified_memory = true;
    gpus[idx].vram_gb = Some(apu_pool_gb); // same as corrected total_ram_gb
}

This ensures model-fit calculations see one coherent memory pool rather than underestimating available resources.

Code Examples: Detection in Practice

Standard Windows Detection (Intel or dGPU System)

let specs = SystemSpecs::detect();
// Uses sysinfo::total_memory() directly
println!("Total RAM: {:.1} GB", specs.total_ram_gb);

APU System With Automatic Correction

let specs = SystemSpecs::detect();
// Internally calls windows_apu_total_ram_gb() for Ryzen AI CPUs
println!("Corrected total RAM: {:.1} GB", specs.total_ram_gb);
println!("Unified memory: {}", specs.unified_memory); // true
println!("GPU VRAM: {:.1} GB", specs.gpu_vram_gb.unwrap()); // same value

Testing the Correction Directly

use llmfit_core::hardware;

let corrected = hardware::windows_apu_total_ram_gb(
    "AMD Ryzen AI MAX+ 395",  // triggers WMI query
    24.0                       // OS-reported fallback
);
println!("Physical RAM: {:.1} GB", corrected); // e.g., 32.0

Key Implementation Files

Function	File Location	Purpose
SystemSpecs::detect()	llmfit-core/src/hardware.rs L71-L100	Orchestrates detection pipeline
windows_apu_total_ram_gb()	llmfit-core/src/hardware.rs L102-L108	Decision logic for carve-out bypass
detect_windows_physical_total_ram_gb()	llmfit-core/src/hardware.rs L150-L170	WMI query implementation
is_amd_unified_memory_apu()	llmfit-core/src/hardware.rs L390-L398	CPU name heuristic matching
GPU APU handling	llmfit-core/src/hardware.rs L201-L215	Sets unified_memory and VRAM linkage

Performance and Reliability Considerations

• WMI query latency: Typically 5-20 ms on warm systems; negligible in startup path
• Fallback behavior: If WMI fails or CPU doesn't match the heuristic, the OS-reported value is used without panic
• Scope limitation: Currently targets AMD Ryzen AI APUs only; Intel Meteor Lake and Lunar Lake unified-memory implementations may require future expansion of is_amd_unified_memory_apu()

Summary

• sysinfo::total_memory() underreports on Windows APU systems due to BIOS GPU carve-out
• AMD Ryzen AI detection triggers a secondary query via windows_apu_total_ram_gb()
• WMI Win32_PhysicalMemory provides DIMM-level capacity independent of BIOS reservations
• Unified memory flag collapses RAM/VRAM distinction for accurate model-fit scoring
• All correction logic resides in llmfit-core/src/hardware.rs with clear fallbacks for non-APU hardware

Frequently Asked Questions

What is a BIOS GPU carveout and why does it hide memory?

A BIOS GPU carveout is a firmware-level memory reservation where the BIOS sets aside a fixed portion of system RAM for the integrated GPU before the operating system boots. Windows only sees and reports the remaining memory, causing applications to underestimate available resources even though the full DIMM capacity remains physically installed.

Does llmfit's APU detection work on Intel processors with unified memory?

No. As implemented in is_amd_unified_memory_apu() at hardware.rs#L390-L398, the heuristic specifically matches AMD Ryzen AI model strings. Intel Meteor Lake and Lunar Lake chips with unified memory architecture would report incorrect RAM figures until the detection pattern is expanded.

What happens if the WMI query fails or returns no results?

The windows_apu_total_ram_gb() function uses unwrap_or(reported_gb), meaning it transparently falls back to the original OS-reported memory value. No error propagates to the user; the tool simply operates with potentially conservative memory estimates rather than crashing or panicking.

How does this affect model loading decisions on APU systems?

Without the correction, llmfit would reject models that actually fit in the full memory pool. By restoring the physical DIMM capacity, the tool correctly computes that a 32 GB APU with 8 GB carveout can still load a 28 GB parameter model—since the GPU can dynamically release carveout space back to the CPU when not actively rendering.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}