How llmfit Discovers Ollama, MLX, llama.cpp, Docker Model Runner, LM Studio, vLLM, and RamaLama via the `which` Crate

The llmfit project leverages the Rust which crate in llmfit-core/src/providers.rs to detect installed LLM runtime binaries across Linux, macOS, and Windows by checking the system PATH without spawning shell commands.

The llmfit open-source project by AlexsJones provides a unified interface for running large language models across multiple backends. At the core of this capability is a lightweight detection system that determines which inference engines are available on the host system. This article examines how llmfit-core/src/providers.rs implements cross-platform binary discovery using the which crate to locate executables for MLX, llama.cpp, Docker Model Runner, and related runtimes.

Cross-Platform Binary Detection

The detection logic centers on a helper function that queries the operating system's PATH environment variable directly through the which crate. This approach avoids Unix-specific shell commands and ensures compatibility with Windows systems.

The binary_exists Helper

In llmfit-core/src/providers.rs at lines 1791-1794, the binary_exists function wraps the which crate's search functionality:

/// Cross‑platform: uses the `which` crate rather than shelling out to a
/// Unix‑only `which` command, so it works on Windows too.
fn binary_exists(name: &str) -> bool {
    which::which(name).is_ok()
}

This function returns true if the specified executable name exists in the system PATH, enabling providers to verify runtime availability before attempting model operations.

The find_binary Wrapper

To obtain the actual filesystem path of discovered binaries, the code provides a find_binary wrapper around lines 1814-1820:

fn find_binary(name: &str) -> Option<String> {
    if binary_exists(name) {
        which::which(name).ok().and_then(|p| p.into_os_string().into_string().ok())
    } else {
        None
    }
}

This utility returns the full path as a String when the binary exists, or None if the executable cannot be located.

Provider-Specific Discovery Mechanisms

Each LLM backend in llmfit uses these helpers differently depending on whether the provider requires local binaries or relies on network-based detection.

MLX Provider Detection

The MLX provider checks for the Hugging Face CLI binary to pull and manage models. During initialization in MlxProvider::start_pull, the code calls find_binary("hf") to verify that the hf command-line tool is installed before attempting model downloads.

llama.cpp Provider Detection

For llama.cpp, the system must locate both the client and server executables. The LlamaCppProvider::default() implementation calls find_binary("llama-cli") and find_binary("llama-server") to detect these binaries separately, ensuring the full llama.cpp toolchain is available.

Docker Model Runner and RamaLama

The Docker Model Runner provider uses find_binary("docker") to verify Docker daemon accessibility. RamaLama detection shares this same binary check, as RamaLama operates as a Docker Model Runner variant identified by the owned_by: "docker" field in model metadata responses.

Network-Based Detection Methods

Some providers do not require local binaries and instead rely on HTTP endpoint probes:

• Ollama: Detected by querying the /api/tags endpoint rather than checking for a binary
• LM Studio: Identified through its native /api/v0/models endpoint
• vLLM: Discovered by inspecting the owned_by field in OpenAI-compatible /v1/models responses

Implementation Examples

The following patterns demonstrate how llmfit integrates binary detection into provider logic:

// Check if llama.cpp CLI is available before model execution
if binary_exists("llama-cli") {
    println!("llama-cli detected – llama.cpp models can be loaded");
}

// Retrieve full path for MLX Hugging Face CLI
let hf_path = find_binary("hf").expect("hf CLI not found in PATH");

For the Docker Model Runner initialization:

// Verify Docker is installed before attempting container operations
if find_binary("docker").is_none() {
    eprintln!("Docker not found – cannot use Docker Model Runner");
}

Summary

• binary_exists in llmfit-core/src/providers.rs (lines 1791-1794) provides the core PATH lookup using the which crate
• find_binary (lines 1814-1820) returns full filesystem paths for located executables
• MLX requires the hf binary detected via find_binary("hf")
• llama.cpp checks for both llama-cli and llama-server binaries
• Docker Model Runner and RamaLama both verify the docker binary is present
• Ollama, LM Studio, and vLLM use network endpoint detection instead of binary checks
• The implementation works cross-platform on Linux, macOS, and Windows without shell dependencies

Frequently Asked Questions

How does llmfit detect MLX availability?

The MLX provider calls find_binary("hf") to locate the Hugging Face CLI tool. This helper function uses the which crate to search the system PATH for the hf executable, returning None if the tool is not installed.

Why does llmfit use the which crate instead of shell commands?

The which crate provides a cross-platform solution that works on Linux, macOS, and Windows without spawning shell processes. Shelling out to Unix-specific which commands would break Windows compatibility and introduce unnecessary overhead.

Which providers require network detection instead of binary checks?

Ollama relies on HTTP requests to /api/tags, LM Studio queries /api/v0/models, and vLLM inspects OpenAI-compatible /v1/models endpoints. These providers operate as background services rather than command-line tools, making binary detection unnecessary.

How does llmfit handle llama.cpp server and CLI binaries?

The LlamaCppProvider checks for both llama-cli and llama-server separately using distinct find_binary calls. This dual verification ensures that both the interactive CLI tool and the server daemon are available for different usage modes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}