# Which LLM Runtime Providers Are Supported by llmfit? A Complete Guide

> Discover which LLM runtime providers llmfit supports including Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio. Effortlessly switch backends with the TUI picker.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: deep-dive
- Published: 2026-08-19

---

**llmfit supports five LLM runtime providers: Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio, automatically detecting available backends and allowing users to switch between them via a TUI picker.**

The `AlexsJones/llmfit` repository provides a unified interface for local and remote large language model inference. At the heart of this system lies the `Provider` enum defined in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs), which abstracts detection, health checking, and model pulling operations across diverse runtime backends.

## Supported LLM Runtime Providers

llmfit integrates with both daemon-based servers and command-line runtimes. Each provider implementation handles endpoint detection, model metadata retrieval, and download orchestration through standardized interfaces.

### Ollama

**Ollama** serves as the primary daemon-based provider, exposing a REST API at `http://localhost:11434` by default. The detection logic in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) queries `GET /api/tags` to enumerate installed models and verify connectivity. When downloading models, llmfit sends `POST /api/pull` requests to the Ollama daemon.

### llama.cpp

**llama.cpp** support targets GGUF-based models, requiring either `llama-cli` or `llama-server` binaries in the system `PATH`. Unlike daemon providers, this backend can download models directly from Hugging Face repositories. Set the `LLAMA_CPP_PATH` environment variable to specify custom binary locations.

### MLX

**MLX** provides Apple Silicon-optimized inference using the `mlx-community` model cache. This provider maps model identifiers to `mlx-community/*` Hugging Face repositories and requires macOS with Apple Silicon hardware. Detection verifies the presence of the MLX Python package and compatible system architecture.

### Docker Model Runner

**Docker Model Runner** leverages Docker Desktop’s built-in model serving feature at `http://localhost:12434` by default. The discovery mechanism queries `GET /engines` to determine available models, while installation uses the `docker model pull` CLI command. This provider enables containerized inference environments.

### LM Studio

**LM Studio** integration connects to the application's local REST API at `http://127.0.0.1:1234`. The provider supports authenticated remote instances via the `LMSTUDIO_API_KEY` environment variable. Detection validates the `/v1/models` endpoint responsiveness.

## How llmfit Discovers and Manages Providers

The provider lifecycle follows three distinct phases implemented in the core Rust modules.

### Automatic Discovery

On startup, llmfit probes each provider's health endpoint to determine availability. The [`providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/providers.rs) module executes parallel health checks: `GET /api/tags` for Ollama, `GET /engines` for Docker Model Runner, and process detection for llama.cpp binaries. Only responsive providers appear in the TUI model listings.

### Model Installation Workflows

Selecting **Download** (`d`) in the interface triggers provider-specific pull mechanisms:

- **Ollama**: Streaming `POST /api/pull` with progress callbacks
- **Docker Model Runner**: Shell execution of `docker model pull <model>`
- **llama.cpp**: Direct Hugging Face GGUF downloads to the cache directory
- **MLX**: Repository cloning from `mlx-community` organizations
- **LM Studio**: API-mediated downloads through the LM Studio backend

### Remote Instance Configuration

Environment variables override default localhost endpoints, enabling network-distributed inference:

```bash

# Remote Ollama server

OLLAMA_HOST="http://192.168.1.100:11434" llmfit fit --perfect

# Custom Docker Model Runner endpoint

DOCKER_MODEL_RUNNER_HOST="http://my-docker-host:12434" llmfit

# Authenticated LM Studio access

LMSTUDIO_HOST="http://192.168.1.200:1234" LMSTUDIO_API_KEY="secret" llmfit

# Custom llama.cpp binary path

LLAMA_CPP_PATH="/opt/llama_cpp" llmfit --cli search "llama"

```

## Switching Providers in the llmfit TUI

When multiple providers can serve a specific model, llmfit defers the runtime selection to the user interface layer defined in [`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs).

Pressing **`d`** on any model row opens the provider-picker modal, displaying compatible runtimes with their current health status. The selection persists for the current session, routing all subsequent inference requests through the chosen backend. This architecture allows A/B testing between MLX and llama.cpp on Apple Silicon, or comparing Ollama versus Docker Model Runner latency.

## Summary

- **Five native providers**: Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio
- **Automatic detection**: Health endpoint probing occurs at startup via [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs)
- **Unified interface**: The `Provider` enum abstracts download, inference, and metadata operations
- **Remote capable**: Environment variables (`OLLAMA_HOST`, `DOCKER_MODEL_RUNNER_HOST`, `LMSTUDIO_HOST`) enable non-local instances
- **Runtime switching**: Press `d` in the TUI to open the provider picker modal when multiple backends support a model

## Frequently Asked Questions

### How does llmfit prioritize providers when multiple options are available?

When several providers can execute a model, llmfish displays all compatible options in the TUI provider picker rather than enforcing a default hierarchy. Users must explicitly select their preferred runtime by pressing `d` on the model row, ensuring intentional backend selection based on latency, quantization quality, or hardware availability.

### Can I use llmfit with remote Ollama or LM Studio instances?

Yes. Set the `OLLAMA_HOST` environment variable for remote Ollama servers, or `LMSTUDIO_HOST` and `LMSTUDIO_API_KEY` for network-accessible LM Studio installations. The detection logic in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) uses these variables when probing health endpoints, treating remote instances identically to local daemons.

### Why is MLX only available on Apple Silicon devices?

MLX requires Apple's Metal Performance Shaders and the unified memory architecture present in M-series chips. The provider detection code specifically checks for `target_os = "macos"` and ARM64 architecture before enabling MLX options, as the underlying `mlx-community` models depend on framework primitives unavailable on x86 or non-Apple platforms.

### How do I add a custom llama.cpp binary location?

Export the `LLAMA_CPP_PATH` environment variable pointing to the directory containing your `llama-cli` or `llama-server` executables. The provider initialization code appends this path to the process search path before attempting binary detection, allowing usage of manually compiled or nightly builds outside standard system directories.