How llmfit Detects and Manages Apple Silicon Unified Memory: A Deep Dive into the Rust Implementation

Apple Silicon unified memory detection in llmfit works by parsing system_profiler output for "Apple M" or "Apple GPU" strings, then treating total system RAM as shared GPU VRAM and enabling Metal-specific memory queries.

llmfit is an open-source Rust framework for fitting large language models to available hardware. One of its key capabilities is correctly identifying Apple Silicon Macs and accounting for their unified memory architecture (UMA), where CPU and GPU share a single memory pool. This article examines exactly how the codebase detects, flags, and leverages unified memory across hardware detection, fit planning, and API exposure.

How llmfit Detects Apple Silicon Hardware

Parsing system_profiler for Apple GPUs

The detection process begins in llmfit-core/src/hardware.rs. The function detect_apple_gpu(total_ram_gb: f64) -> Option<f64> executes system_profiler SPDisplaysDataType and scans the output:

// From llmfit-core/src/hardware.rs, lines 1595-1625
fn detect_apple_gpu(total_ram_gb: f64) -> Option<f64> {
    // Executes: system_profiler SPDisplaysDataType
    // Searches for lines containing "Apple M" or "Apple GPU"
    // Returns Some(total_ram_gb) when found, None otherwise
}

When detect_apple_gpu finds "Apple M" or "Apple GPU" in the profiler output, it returns the total system RAM as the GPU's VRAM value. This works because on Apple Silicon, there is no discrete VRAM—the entire memory pool is shared.

Building the GPU Information Structure

All detected GPUs are collected in detect_all_gpus. For Apple Silicon devices, a GpuInfo struct is populated with unified_memory: true:

GpuInfo Field	Apple Silicon Value
name	e.g., "Apple M3 Pro"
vram_gb	total_ram_gb of the system
backend	GpuBackend::Metal
unified_memory	true

Propagating the Unified Memory Flag to SystemSpecs

Extraction in SystemSpecs::detect()

The SystemSpecs::detect() method extracts the primary GPU (the one with the most VRAM) and propagates its unified_memory flag to the top-level system specification:

// From llmfit-core/src/hardware.rs, lines 1005-1010
let primary = gpus.first();
// ...
let unified_memory = primary.map(|g| g.unified_memory).unwrap_or(false);

This boolean becomes a cornerstone for downstream logic throughout llmfit.

Structure of Detected System Information

The resulting SystemSpecs structure contains these memory-related fields:

pub struct SystemSpecs {
    pub total_ram_gb: f64,        // Physical system RAM (also VRAM on Apple Silicon)
    pub available_ram_gb: f64,    // RAM free for CPU-only inference
    pub gpu_vram_gb: Option<f64>, // Some(total_ram_gb) on Apple Silicon
    pub unified_memory: bool,     // true only for Apple Silicon / UMA
    pub backend: GpuBackend,      // GpuBackend::Metal for Apple Silicon
    pub gpu_available_gb: Option<f64>, // Metal API query result, Apple Silicon only
}

Querying Available GPU Memory on Apple Silicon

Conditional Metal API Calls

Because unified memory is static (it does not shrink with current usage like discrete GPU VRAM), llmfit only queries GPU-available memory via the Metal API when unified_memory is true. The helper detect_gpu_available_gb() handles this:

// From llmfit-core/src/hardware.rs, lines 1335-1340
let gpu_available_gb = if unified_memory {
    detect_gpu_available_gb()
} else {
    None
};

This avoids unnecessary API calls on non-Apple systems and ensures accurate memory accounting where the unified pool matters.

How llmfit Uses Unified Memory Detection

Skipping CPU Offload Paths

In llmfit-core/src/fit.rs, the fit calculation skips CPU-offload paths when specs.unified_memory is true. There is no separate RAM pool to spill to—offloading from GPU to CPU would use the same memory, providing no benefit.

Memory Planning with Shared Pool Awareness

The planning logic in llmfit-core/src/plan.rs respects the unified_memory flag when estimating memory requirements. Model-size checks use the shared pool rather than treating VRAM and RAM as distinct resources, preventing false-negative fit rejections.

API Exposure for Frontend Integration

The HTTP API in llmfit-tui/src/serve_api.rs exposes unified_memory and gpu_available_gb to callers, enabling frontends to display accurate memory information:

// From llmfit-tui/src/serve_api.rs
let is_apple_silicon = specs.backend == GpuBackend::Metal && specs.unified_memory;

Practical Code Examples

Example 1: Detecting and Displaying Unified Memory Status

use llmfit_core::hardware::SystemSpecs;

fn main() {
    let specs = SystemSpecs::detect();
    if specs.unified_memory {
        println!(
            "Apple Silicon detected: unified memory = {:.2} GB (GPU & CPU share this pool)",
            specs.total_ram_gb
        );
    } else {
        println!("No unified memory; discrete GPU detected.");
    }
}

Example 2: Custom Planner Respecting Unified Memory

use llmfit_core::{hardware::SystemSpecs, plan::Plan};

fn plan_for_model(model_params: u64) {
    let specs = SystemSpecs::detect();
    
    // On Apple Silicon, compare model size against total RAM,
    // not against a separate VRAM limit
    let max_mem = if specs.unified_memory {
        specs.total_ram_gb
    } else {
        specs.gpu_vram_gb.unwrap_or(0.0)
    };

    let model_size_gb = model_params as f64 * 0.5 / 1024.0 / 1024.0 / 1024.0;
    
    if model_size_gb > max_mem {
        eprintln!("Model too large for this system.");
    } else {
        let plan = Plan::new(&specs, model_params);
        println!("Plan: {:?}", plan);
    }
}

Example 3: Querying the API for Unified Memory Information

curl http://localhost:8080/api/v1/specs | jq '.unified_memory, .gpu_available_gb'

Typical output on an Apple Silicon Mac:

true
42.0

Key Source Files

File	Purpose	Lines of Interest
llmfit-core/src/hardware.rs	Core hardware detection, Apple Silicon logic	1595-1625 (detect_apple_gpu), 1005-1010 (flag propagation), 1335-1340 (GPU available query)
llmfit-core/src/fit.rs	Model fitting with unified-memory awareness	CPU-offload skip logic
llmfit-core/src/plan.rs	Memory planning respecting unified_memory	Model size estimation
llmfit-tui/src/serve_api.rs	HTTP API exposing memory flags	API response construction

Summary

• detect_apple_gpu in hardware.rs identifies Apple Silicon by parsing system_profiler for "Apple M" or "Apple GPU" strings
• Total system RAM becomes both CPU RAM and GPU VRAM when unified memory is detected
• unified_memory: true propagates from GpuInfo through SystemSpecs to enable Metal-specific behavior
• gpu_available_gb is only queried via Metal API when unified memory is active
• CPU-offload paths are skipped for unified memory systems since spilling provides no benefit
• The API exposes unified_memory and gpu_available_gb for frontend display and logic

Frequently Asked Questions

How does llmfit distinguish Apple Silicon from Intel Macs with AMD GPUs?

llmfit executes system_profiler SPDisplaysDataType and searches the output for strings containing "Apple M" or "Apple GPU". Intel Macs with AMD GPUs will show "AMD Radeon" or similar identifiers instead, causing detect_apple_gpu to return None and leaving unified_memory as false.

Why does gpu_available_gb only exist for Apple Silicon systems?

On discrete GPUs, available VRAM fluctuates with usage and is difficult to query reliably cross-platform. On Apple Silicon, unified memory is static—the total pool size never changes. llmfit therefore only calls detect_gpu_available_gb() when unified_memory is true, using the Metal API for an accurate, meaningful reading.

Does llmfit support other unified memory architectures besides Apple Silicon?

The unified_memory boolean in GpuInfo and SystemSpecs is architecture-agnostic. While the current implementation only sets it to true for Apple Silicon via detect_apple_gpu, the flag could be extended to other UMA systems (certain ARM SoCs, future Intel/AMD APUs) by adding detection logic in hardware.rs without changing downstream consumers.

Can I force unified memory mode on non-Apple systems?

There is no configuration flag to override the hardware-detected unified_memory value in llmfit. The field is strictly derived from system_profiler output. This design prevents misconfiguration that would lead to incorrect memory planning and failed model loads.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}