# Using llmfit with MLX on Apple Silicon: Setup and Configuration Guide

> Learn how to use llmfit with MLX on Apple Silicon. This guide details setup and configuration for seamless MLX inference on compatible hardware.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: getting-started
- Published: 2026-08-19

---

**llmfit supports MLX inference exclusively on Apple Silicon devices that expose the Metal GPU backend and unified memory architecture, automatically detecting compatible hardware in [`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs) and filtering out MLX-only models on incompatible platforms.**

The **llmfit** repository provides a Rust-based CLI tool for benchmarking large language model inference performance across different hardware configurations. For developers working on Apple Silicon Macs, llmfit integrates natively with Apple's **MLX** machine learning framework to leverage GPU acceleration, but this functionality is gated by strict hardware detection logic to ensure compatibility.

## Hardware Requirements and Auto-Detection

MLX support in llmfit is not merely a configuration option—it is a hardware-gated feature. According to the source code in [`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs), the system initializes the MLX runtime only when two specific conditions are met: the presence of `GpuBackend::Metal` and the availability of unified memory. These checks occur during the `is_apple_silicon` flag computation in [`llmfit-tui/src/main.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/main.rs).

When the tool detects an incompatible environment (Linux, Windows, or Intel-based Macs without Metal support), it explicitly filters out MLX-quantized models from the results. This prevents users from attempting to load models that require the MLX runtime on unsupported hardware. The detection logic ensures that **MLX-only models never appear in the fit results** on non-Apple Silicon systems.

## How llmfit Discovers MLX Models

The model discovery process relies on a specific naming convention and local cache inspection. As implemented in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs), the `MlxProvider` scans the local Hugging Face cache directory for repositories containing the **"-MLX"** suffix in their names (e.g., `Qwen3-8B-MLX-4bit`).

When the provider identifies these models, it sets the `is_mlx_only` flag defined in [`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs) and assigns the **`InferenceRuntime::Mlx`** variant to their runtime field. This metadata tagging ensures that the fit engine routes these specific models through the MLX inference path while maintaining compatibility with standard PyTorch or GGUF models in the same workspace.

## Configuring the MLX Runtime

By default, llmfit communicates with a local MLX server running at `http://localhost:8080`. You can override this endpoint using the **`MLX_LM_HOST`** environment variable, which the `MlxProvider::default()` constructor reads during initialization (lines 45-55 in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs)).

The runtime selection respects explicit user overrides through the CLI. When you specify `--force-runtime mlx`, the argument parser in [`llmfit-tui/src/main.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/main.rs) (lines 13-28) maps the string "mlx" to the `InferenceRuntime::Mlx` enum variant, bypassing auto-detection and forcing the fit engine to use the `MlxProvider` implementation regardless of other available backends.

## CLI Commands for MLX Workflows

### Force MLX Runtime on Apple Silicon

To explicitly run a performance fit using the MLX backend on a compatible Mac:

```bash
cargo run -- fit --force-runtime mlx

```

This command triggers the `MlxProvider::detect_with_installed` method (lines 26-34 in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs)) to establish a connection to the local MLX server and compute throughput estimates using Metal GPU acceleration.

### Filter for MLX-Compatible Models Only

When browsing available models, restrict the output to those supporting MLX quantization:

```bash
cargo run -- list --runtime mlx

```

The runtime filter logic in [`llmfit-tui/src/main.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/main.rs) (lines 60-66) retains only entries where the `runtime` field equals `InferenceRuntime::Mlx`, hiding GGUF or PyTorch variants that cannot execute on the MLX framework.

### Override the MLX Server Endpoint

If your MLX server runs on a non-standard port or remote host within your network:

```bash
export MLX_LM_HOST=http://127.0.0.1:8001
cargo run -- fit --force-runtime mlx

```

The provider falls back to `http://localhost:8080` only if the `MLX_LM_HOST` variable is unset or contains a malformed URL.

## Platform Restrictions and Filtering

The enforcement of Apple Silicon exclusivity happens at the application layer, not just the provider level. In [`llmfit-tui/src/main.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/main.rs), the application computes the `is_apple_silicon` boolean flag early in the execution lifecycle. If this flag evaluates to false, the system applies a pre-filter to the model registry that removes all entries marked with `Model::is_mlx_only` before displaying them to the user.

This design prevents runtime errors and confusing failures that would occur if a user attempted to load MLX-quantized weights on CUDA or CPU-only environments. The UI components in [`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs) display the MLX detection status to confirm when the accelerated path is active.

## Summary

- **Apple Silicon Required**: llmfit enables MLX support only on macOS systems with Metal GPU backend and unified memory, as verified in [`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs).
- **Model Tagging**: MLX models are identified by the "-MLX" suffix in their Hugging Face repository names and flagged with `is_mlx_only` in [`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs).
- **Server Communication**: The `MlxProvider` queries `http://localhost:8080` by default, configurable via the `MLX_LM_HOST` environment variable.
- **CLI Overrides**: Use `--force-runtime mlx` to bypass auto-detection, or `--runtime mlx` to filter model lists.
- **Cross-Platform Safety**: Non-Apple Silicon systems automatically hide MLX-only models to prevent incompatible execution attempts.

## Frequently Asked Questions

### Can I use llmfit with MLX on Windows or Linux?

No. According to the source code in [`llmfit-tui/src/main.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/main.rs), llmfit explicitly filters out MLX-only models on non-Apple Silicon platforms. The MLX runtime requires the Metal GPU backend (`GpuBackend::Metal`) and unified memory architecture found exclusively in Apple Silicon Macs. On Windows or Linux, llmfit will not display or load models with the "-MLX" suffix, even if you attempt to force the runtime manually.

### What naming convention do MLX models need in llmfit?

MLX-compatible models must include **"-MLX"** in their Hugging Face repository name (for example, `Qwen3-8B-MLX-4bit`). The `MlxProvider` implementation in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) scans the local cache for this substring to identify compatible checkpoints. Models without this suffix are treated as standard PyTorch or GGUF formats and routed to their respective inference runtimes.

### Does llmfit require a running MLX server before execution?

Yes. The `MlxProvider` expects a local MLX server accessible at `http://localhost:8080` by default. You can verify connectivity or specify an alternative endpoint using the `MLX_LM_HOST` environment variable. If the server is unreachable, the `detect_with_installed` method will fail to compute throughput estimates, and the fit operation will exit with a connection error rather than falling back to CPU execution.

### How does llmfit handle mixed hardware environments?

On systems that meet the Apple Silicon criteria (Metal backend + unified memory), llmfit displays both MLX and non-MLX models, allowing you to benchmark across different quantization schemes. However, the application computes the `is_apple_silicon` flag at startup and applies a filter in the TUI layer that removes MLX-only entries from the model list on incompatible hardware, ensuring that users cannot accidentally attempt to load Metal-specific weights on CUDA or CPU-only configurations.