Which LLM Runtime Providers Are Supported by llmfit? A Complete Guide

llmfit supports five LLM runtime providers: Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio, automatically detecting available backends and allowing users to switch between them via a TUI picker.

The AlexsJones/llmfit repository provides a unified interface for local and remote large language model inference. At the heart of this system lies the Provider enum defined in llmfit-core/src/providers.rs, which abstracts detection, health checking, and model pulling operations across diverse runtime backends.

Supported LLM Runtime Providers

llmfit integrates with both daemon-based servers and command-line runtimes. Each provider implementation handles endpoint detection, model metadata retrieval, and download orchestration through standardized interfaces.

Ollama

Ollama serves as the primary daemon-based provider, exposing a REST API at http://localhost:11434 by default. The detection logic in llmfit-core/src/providers.rs queries GET /api/tags to enumerate installed models and verify connectivity. When downloading models, llmfit sends POST /api/pull requests to the Ollama daemon.

llama.cpp

llama.cpp support targets GGUF-based models, requiring either llama-cli or llama-server binaries in the system PATH. Unlike daemon providers, this backend can download models directly from Hugging Face repositories. Set the LLAMA_CPP_PATH environment variable to specify custom binary locations.

MLX

MLX provides Apple Silicon-optimized inference using the mlx-community model cache. This provider maps model identifiers to mlx-community/* Hugging Face repositories and requires macOS with Apple Silicon hardware. Detection verifies the presence of the MLX Python package and compatible system architecture.

Docker Model Runner

Docker Model Runner leverages Docker Desktop’s built-in model serving feature at http://localhost:12434 by default. The discovery mechanism queries GET /engines to determine available models, while installation uses the docker model pull CLI command. This provider enables containerized inference environments.

LM Studio

LM Studio integration connects to the application's local REST API at http://127.0.0.1:1234. The provider supports authenticated remote instances via the LMSTUDIO_API_KEY environment variable. Detection validates the /v1/models endpoint responsiveness.

How llmfit Discovers and Manages Providers

The provider lifecycle follows three distinct phases implemented in the core Rust modules.

Automatic Discovery

On startup, llmfit probes each provider's health endpoint to determine availability. The providers.rs module executes parallel health checks: GET /api/tags for Ollama, GET /engines for Docker Model Runner, and process detection for llama.cpp binaries. Only responsive providers appear in the TUI model listings.

Model Installation Workflows

Selecting Download (d) in the interface triggers provider-specific pull mechanisms:

• Ollama: Streaming POST /api/pull with progress callbacks
• Docker Model Runner: Shell execution of docker model pull <model>
• llama.cpp: Direct Hugging Face GGUF downloads to the cache directory
• MLX: Repository cloning from mlx-community organizations
• LM Studio: API-mediated downloads through the LM Studio backend

Remote Instance Configuration

Environment variables override default localhost endpoints, enabling network-distributed inference:

# Remote Ollama server

OLLAMA_HOST="http://192.168.1.100:11434" llmfit fit --perfect

# Custom Docker Model Runner endpoint

DOCKER_MODEL_RUNNER_HOST="http://my-docker-host:12434" llmfit

# Authenticated LM Studio access

LMSTUDIO_HOST="http://192.168.1.200:1234" LMSTUDIO_API_KEY="secret" llmfit

# Custom llama.cpp binary path

LLAMA_CPP_PATH="/opt/llama_cpp" llmfit --cli search "llama"

Switching Providers in the llmfit TUI

When multiple providers can serve a specific model, llmfit defers the runtime selection to the user interface layer defined in llmfit-tui/src/tui_ui.rs.

Pressing d on any model row opens the provider-picker modal, displaying compatible runtimes with their current health status. The selection persists for the current session, routing all subsequent inference requests through the chosen backend. This architecture allows A/B testing between MLX and llama.cpp on Apple Silicon, or comparing Ollama versus Docker Model Runner latency.

Summary

• Five native providers: Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio
• Automatic detection: Health endpoint probing occurs at startup via llmfit-core/src/providers.rs
• Unified interface: The Provider enum abstracts download, inference, and metadata operations
• Remote capable: Environment variables (OLLAMA_HOST, DOCKER_MODEL_RUNNER_HOST, LMSTUDIO_HOST) enable non-local instances
• Runtime switching: Press d in the TUI to open the provider picker modal when multiple backends support a model

Frequently Asked Questions

How does llmfit prioritize providers when multiple options are available?

When several providers can execute a model, llmfish displays all compatible options in the TUI provider picker rather than enforcing a default hierarchy. Users must explicitly select their preferred runtime by pressing d on the model row, ensuring intentional backend selection based on latency, quantization quality, or hardware availability.

Can I use llmfit with remote Ollama or LM Studio instances?

Yes. Set the OLLAMA_HOST environment variable for remote Ollama servers, or LMSTUDIO_HOST and LMSTUDIO_API_KEY for network-accessible LM Studio installations. The detection logic in llmfit-core/src/providers.rs uses these variables when probing health endpoints, treating remote instances identically to local daemons.

Why is MLX only available on Apple Silicon devices?

MLX requires Apple's Metal Performance Shaders and the unified memory architecture present in M-series chips. The provider detection code specifically checks for target_os = "macos" and ARM64 architecture before enabling MLX options, as the underlying mlx-community models depend on framework primitives unavailable on x86 or non-Apple platforms.

How do I add a custom llama.cpp binary location?

Export the LLAMA_CPP_PATH environment variable pointing to the directory containing your llama-cli or llama-server executables. The provider initialization code appends this path to the process search path before attempting binary detection, allowing usage of manually compiled or nightly builds outside standard system directories.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}