# How llmfit Handles Unified Memory on Apple Silicon: Detection and Optimization Strategy

> Discover how llmfit masters unified memory on Apple Silicon. Learn its detection methods and optimization strategies for efficient CPU and GPU inference.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: internals
- Published: 2026-07-23

---

**llmfit detects Apple Silicon by parsing `system_profiler` output, marks the system as `unified_memory: true`, and treats the total system RAM as a single shared pool for both CPU and GPU inference, disabling CPU-offload paths and selecting quantization levels that fit within the shared memory budget.**

The `llmfit` repository (AlexsJones/llmfit) implements specialized logic to handle the unified memory architecture of Apple Silicon devices (M1/M2/M3). Unlike discrete GPU systems where CPU and GPU memory are physically separate, Apple Silicon shares a single memory pool between the CPU and GPU, requiring distinct detection and allocation strategies.

## Hardware Detection and Apple Silicon Identification

The detection pipeline begins in [`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs) within the `SystemSpecs::detect()` method. When initializing system specifications, `llmfit` calls `detect_apple_gpu()` to determine if the current device utilizes Apple Silicon architecture.

This function executes `system_profiler SPDisplaysDataType` and scans the output for lines containing **"Apple M"** or **"Apple GPU"** (lines 72-80). Upon detecting an Apple GPU, the system retrieves the total system RAM (`total_ram_gb`) and assigns this value to the GPU's VRAM field while setting the `unified_memory` flag to `true`.

The resulting `SystemSpecs` struct contains:
- `unified_memory: true`
- `gpu_vram_gb: Some(total_ram_gb)` 
- `backend: GpuBackend::Metal`

## Unified Memory Representation and Pool Management

On Apple Silicon, the traditional distinction between system RAM and video RAM does not exist. In [`hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/hardware.rs) (lines 87-94), `llmfit` represents this architecture by reporting the total RAM capacity as the available GPU memory. This ensures that subsequent memory calculations treat the entire physical memory as a single allocatable pool.

When `unified_memory` is enabled, `SystemSpecs::detect()` also invokes `detect_gpu_available_gb()` (lines 27-31) to query Metal's `recommendedMaxWorkingSetSize`. This provides a precise measurement of the currently available unified memory, accounting for system overhead and existing allocations.

## Model Fit Analysis and CPU Offload Disabling

The fit determination logic resides in [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs) within the `ModelFit::analyze()` routine. When processing Apple Silicon systems, the code explicitly checks `system.unified_memory` to alter the standard inference path.

Because the CPU and GPU share the same physical memory pool, the **CPU-offload** mechanism—which typically spills layers from VRAM to system RAM—is disabled (lines 18-22). There is no separate memory pool to offload to, making such strategies redundant and potentially counterproductive.

Instead, `ModelFit::analyze()` utilizes the unified VRAM size (`system.gpu_vram_gb`) as the constraint for both GPU and CPU inference scenarios. The system selects the optimal quantization level that fits within this shared pool, adding an internal note: "Unified memory: GPU and CPU share the same pool" (lines 18-28).

## Fit Scoring and Memory Headroom

The `score_fit()` function evaluates model compatibility by comparing `mem_required` against `mem_available` within the unified pool context. Fit levels (Perfect, Good, Marginal, Poor) reflect whether the quantized model can reside within the total Apple Silicon memory budget without exceeding the shared capacity.

This scoring mechanism ensures that users receive accurate predictions about inference feasibility, accounting for the fact that both model weights and activation tensors must coexist in the same physical memory alongside macOS system processes.

## Practical Implementation Examples

The following examples demonstrate how to programmatically inspect unified memory detection and perform fit analysis on Apple Silicon devices.

**Detecting and Displaying System Specifications:**

```rust
let specs = llmfit_core::hardware::SystemSpecs::detect();
println!("Total RAM: {:.1} GB", specs.total_ram_gb);
println!("GPU backend: {:?}", specs.backend);
println!("Unified memory: {}", specs.unified_memory);
if let Some(vram) = specs.gpu_vram_gb {
    println!("GPU VRAM (shared): {:.1} GB", vram);
}

```

**Running Model Fit Analysis:**

```rust
let specs = llmfit_core::hardware::SystemSpecs::detect();
let model = llmfit_core::models::LlmModel::from_name("tinyllama-1b").unwrap();
let fit = llmfit_core::fit::ModelFit::analyze(&model, &specs, None);
println!("Fit level: {:?}", fit.fit_level);
println!("Run mode:   {:?}", fit.run_mode); // Returns RunMode::Gpu on unified memory

```

## TUI Integration and User Interface

The unified memory status is surfaced to users through [`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs) (line 111). The terminal user interface checks `primary.unified_memory` to display a "Unified memory" indicator in the system status bar, providing immediate visual confirmation that the tool has correctly identified the Apple Silicon architecture and is operating in unified memory mode.

## Summary

- **Apple Silicon Detection**: [`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs) executes `system_profiler` and parses for "Apple M" or "Apple GPU" strings to trigger unified memory mode.
- **Shared Pool Representation**: The system reports total RAM as GPU VRAM and sets `unified_memory: true`, using Metal's `recommendedMaxWorkingSetSize` for precise availability queries.
- **Disabled CPU Offload**: [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs) disables CPU-offload paths because no separate memory pool exists for spilling layers.
- **Unified Fit Scoring**: `ModelFit::analyze()` and `score_fit()` evaluate quantization levels against the single shared memory budget.
- **UI Indication**: The TUI displays the unified memory flag to confirm correct hardware detection.

## Frequently Asked Questions

### How does llmfit detect Apple Silicon hardware?

`llmfit` detects Apple Silicon through the `detect_apple_gpu()` function in [`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs). It executes `system_profiler SPDisplaysDataType` and searches the output for lines containing "Apple M" or "Apple GPU". When found, it marks the system with `unified_memory: true` and configures the Metal backend.

### Why does llmfit disable CPU-offload on unified memory systems?

CPU-offload is disabled because Apple Silicon devices utilize a single physical memory pool shared between the CPU and GPU. Traditional offload strategies assume separate VRAM and system RAM, spilling layers from GPU memory to CPU memory. On unified memory architectures, this separation does not exist, making offload redundant.

### How does llmfit calculate available memory on Apple Silicon?

When `unified_memory` is true, `llmfit` calls `detect_gpu_available_gb()` to query Metal's `recommendedMaxWorkingSetSize`. This API provides the recommended maximum memory available for the GPU workload within the unified pool, accounting for current system usage and macOS memory management.

### Can llmfit run on Apple Silicon without Metal?

While `llmfit` can fall back to CPU-only inference if GPU detection fails, the tool is designed to utilize Metal on Apple Silicon. If Metal is unavailable, the system would treat the device as a generic CPU platform, losing the optimizations specific to unified memory detection and shared pool management.