How llmfit Handles Unified Memory on Apple Silicon: Detection and Optimization Strategy

llmfit detects Apple Silicon by parsing system_profiler output, marks the system as unified_memory: true, and treats the total system RAM as a single shared pool for both CPU and GPU inference, disabling CPU-offload paths and selecting quantization levels that fit within the shared memory budget.

The llmfit repository (AlexsJones/llmfit) implements specialized logic to handle the unified memory architecture of Apple Silicon devices (M1/M2/M3). Unlike discrete GPU systems where CPU and GPU memory are physically separate, Apple Silicon shares a single memory pool between the CPU and GPU, requiring distinct detection and allocation strategies.

Hardware Detection and Apple Silicon Identification

The detection pipeline begins in llmfit-core/src/hardware.rs within the SystemSpecs::detect() method. When initializing system specifications, llmfit calls detect_apple_gpu() to determine if the current device utilizes Apple Silicon architecture.

This function executes system_profiler SPDisplaysDataType and scans the output for lines containing "Apple M" or "Apple GPU" (lines 72-80). Upon detecting an Apple GPU, the system retrieves the total system RAM (total_ram_gb) and assigns this value to the GPU's VRAM field while setting the unified_memory flag to true.

The resulting SystemSpecs struct contains:

• unified_memory: true
• gpu_vram_gb: Some(total_ram_gb)
• backend: GpuBackend::Metal

Unified Memory Representation and Pool Management

On Apple Silicon, the traditional distinction between system RAM and video RAM does not exist. In hardware.rs (lines 87-94), llmfit represents this architecture by reporting the total RAM capacity as the available GPU memory. This ensures that subsequent memory calculations treat the entire physical memory as a single allocatable pool.

When unified_memory is enabled, SystemSpecs::detect() also invokes detect_gpu_available_gb() (lines 27-31) to query Metal's recommendedMaxWorkingSetSize. This provides a precise measurement of the currently available unified memory, accounting for system overhead and existing allocations.

Model Fit Analysis and CPU Offload Disabling

The fit determination logic resides in llmfit-core/src/fit.rs within the ModelFit::analyze() routine. When processing Apple Silicon systems, the code explicitly checks system.unified_memory to alter the standard inference path.

Because the CPU and GPU share the same physical memory pool, the CPU-offload mechanism—which typically spills layers from VRAM to system RAM—is disabled (lines 18-22). There is no separate memory pool to offload to, making such strategies redundant and potentially counterproductive.

Instead, ModelFit::analyze() utilizes the unified VRAM size (system.gpu_vram_gb) as the constraint for both GPU and CPU inference scenarios. The system selects the optimal quantization level that fits within this shared pool, adding an internal note: "Unified memory: GPU and CPU share the same pool" (lines 18-28).

Fit Scoring and Memory Headroom

The score_fit() function evaluates model compatibility by comparing mem_required against mem_available within the unified pool context. Fit levels (Perfect, Good, Marginal, Poor) reflect whether the quantized model can reside within the total Apple Silicon memory budget without exceeding the shared capacity.

This scoring mechanism ensures that users receive accurate predictions about inference feasibility, accounting for the fact that both model weights and activation tensors must coexist in the same physical memory alongside macOS system processes.

Practical Implementation Examples

The following examples demonstrate how to programmatically inspect unified memory detection and perform fit analysis on Apple Silicon devices.

Detecting and Displaying System Specifications:

let specs = llmfit_core::hardware::SystemSpecs::detect();
println!("Total RAM: {:.1} GB", specs.total_ram_gb);
println!("GPU backend: {:?}", specs.backend);
println!("Unified memory: {}", specs.unified_memory);
if let Some(vram) = specs.gpu_vram_gb {
    println!("GPU VRAM (shared): {:.1} GB", vram);
}

Running Model Fit Analysis:

let specs = llmfit_core::hardware::SystemSpecs::detect();
let model = llmfit_core::models::LlmModel::from_name("tinyllama-1b").unwrap();
let fit = llmfit_core::fit::ModelFit::analyze(&model, &specs, None);
println!("Fit level: {:?}", fit.fit_level);
println!("Run mode:   {:?}", fit.run_mode); // Returns RunMode::Gpu on unified memory

TUI Integration and User Interface

The unified memory status is surfaced to users through llmfit-tui/src/tui_ui.rs (line 111). The terminal user interface checks primary.unified_memory to display a "Unified memory" indicator in the system status bar, providing immediate visual confirmation that the tool has correctly identified the Apple Silicon architecture and is operating in unified memory mode.

Summary

• Apple Silicon Detection: llmfit-core/src/hardware.rs executes system_profiler and parses for "Apple M" or "Apple GPU" strings to trigger unified memory mode.
• Shared Pool Representation: The system reports total RAM as GPU VRAM and sets unified_memory: true, using Metal's recommendedMaxWorkingSetSize for precise availability queries.
• Disabled CPU Offload: llmfit-core/src/fit.rs disables CPU-offload paths because no separate memory pool exists for spilling layers.
• Unified Fit Scoring: ModelFit::analyze() and score_fit() evaluate quantization levels against the single shared memory budget.
• UI Indication: The TUI displays the unified memory flag to confirm correct hardware detection.

Frequently Asked Questions

How does llmfit detect Apple Silicon hardware?

llmfit detects Apple Silicon through the detect_apple_gpu() function in llmfit-core/src/hardware.rs. It executes system_profiler SPDisplaysDataType and searches the output for lines containing "Apple M" or "Apple GPU". When found, it marks the system with unified_memory: true and configures the Metal backend.

Why does llmfit disable CPU-offload on unified memory systems?

CPU-offload is disabled because Apple Silicon devices utilize a single physical memory pool shared between the CPU and GPU. Traditional offload strategies assume separate VRAM and system RAM, spilling layers from GPU memory to CPU memory. On unified memory architectures, this separation does not exist, making offload redundant.

How does llmfit calculate available memory on Apple Silicon?

When unified_memory is true, llmfit calls detect_gpu_available_gb() to query Metal's recommendedMaxWorkingSetSize. This API provides the recommended maximum memory available for the GPU workload within the unified pool, accounting for current system usage and macOS memory management.

Can llmfit run on Apple Silicon without Metal?

While llmfit can fall back to CPU-only inference if GPU detection fails, the tool is designed to utilize Metal on Apple Silicon. If Metal is unavailable, the system would treat the device as a generic CPU platform, losing the optimizations specific to unified memory detection and shared pool management.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}