# How llmfit Detects Models from Docker Containers: Architecture and Implementation > Discover how llmfit detects models from Docker containers. Learn about its architecture and implementation, including verifying Docker, configuring the DMR client, and probing the /v1/models endpoint. - Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit) - Tags: architecture - Published: 2026-08-20 --- **llmfit uses the Docker Model Runner (DMR) provider to detect containerized models through three steps: verifying Docker Desktop installation, creating a configurable DMR client, and probing the `/v1/models` endpoint to enumerate installed engines.** The **llmfit** project implements sophisticated Docker container model detection as part of its multi-provider AI model management system. This article examines how the `DockerModelRunnerProvider` in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) discovers, enumerates, and prepares Docker-based models for inference, with full source code transparency from the AlexsJones/llmfit repository. ## The Docker Model Runner Provider Architecture `llmfit` abstracts model detection through the `ModelProvider` trait, with `DockerModelRunnerProvider` as its Docker-specific implementation. This design enables pluggable detection for local, remote, and containerized inference engines through a unified interface. The detection workflow separates platform-specific concerns: Docker Desktop presence checks run first to avoid expensive network timeouts, followed by API probing only when the infrastructure is confirmed available. ## Step 1: Verify Docker Desktop Installation Before attempting any network operations, `llmfit` confirms Docker Desktop is installed and running through platform-aware checks. ### Cross-Platform Installation Detection The function `docker_desktop_installed()` in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) scans predefined installation candidates returned by `docker_desktop_install_candidates()`: - **Windows**: `C:\Program Files\Docker\Docker` - **macOS**: `/Applications/Docker.app` - **Linux**: `/opt/docker-desktop`, `/usr/bin/docker-desktop` - **User directory**: `~/.docker/desktop/builds` ```rust // From providers.rs: docker_desktop_install_candidates() // Returns platform-specific paths to check fn docker_desktop_install_candidates() -> Vec { #[cfg(target_os = "windows")] { vec![ PathBuf::from(r"C:\Program Files\Docker\Docker"), // Additional Windows paths... ] } #[cfg(target_os = "macos")] { vec![ PathBuf::from("/Applications/Docker.app"), ] } #[cfg(target_os = "linux")] { vec![ PathBuf::from("/opt/docker-desktop"), PathBuf::from("/usr/bin/docker-desktop"), home_dir().map(|h| h.join(".docker/desktop")).unwrap_or_default(), ] } } ``` ### Linux Runtime Verification On Linux, `is_docker_desktop_running()` provides an additional guard by checking for the Docker Desktop Unix socket at `/run/docker-desktop/docker.sock` or `~/.docker/desktop/docker.sock`. This check prevents the 800 ms HTTP timeout that would occur if Docker Desktop is installed but not running. ## Step 2: Create the DMR Client With Docker Desktop confirmed, `DockerModelRunnerProvider::new()` constructs the provider with configurable connection parameters. ### Configurable Base URL The provider defaults to `http://localhost:12434` (the standard DMR endpoint) but respects the `DOCKER_MODEL_RUNNER_HOST` environment variable for custom deployments: ```rust // Create the provider with automatic environment detection let docker_provider = DockerModelRunnerProvider::new(); // Or with explicit configuration std::env::set_var("DOCKER_MODEL_RUNNER_HOST", "http://docker-host:12434"); let custom_provider = DockerModelRunnerProvider::new(); ``` The `normalize_docker_mr_host()` function in [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) handles URL normalization, ensuring consistent formatting regardless of trailing slashes or protocol specifications. ## Step 3: Probe the DMR API for Installed Models The `detect_with_installed()` method performs the actual model enumeration through a single HTTP GET request to `{base_url}/v1/models`. ### Early Exit for Unavailable Infrastructure On Linux systems, if `is_docker_desktop_running()` returns false, the method returns early with `available = false`, avoiding the network timeout entirely. This optimization is critical for TUI responsiveness. ### Response Processing and Normalization Successful responses deserialize into this structure: ```rust #[derive(Deserialize)] struct DockerModelList { data: Vec, } #[derive(Deserialize)] struct DockerEngine { id: String, // e.g., "llama3.1:8B-Q4_K_M" // Additional metadata... } ``` For each engine, `llmfit` generates multiple identifier variants stored in a `HashSet`: | Variant | Example | Purpose | |---------|---------|---------| | Full ID (lowercased) | `llama3.1:8b-q4_k_m` | Exact matching | | Short name | `8b-q4_k_m` | User-friendly display | | Base name (quantization stripped) | `llama3.1:8b` | Fuzzy matching across formats | ```rust // Detection and enumeration example let (available, models, count) = docker_provider.detect_with_installed(); if available { println!("Docker Model Runner: {} models available", count); // models contains all normalized identifiers for identifier in &models { println!(" - {}", identifier); } } ``` ### Return Value Structure The method returns a three-element tuple used throughout the application: - `available: bool` — Whether DMR is reachable and responding - `set: HashSet` — All normalized model identifiers for membership testing - `count: usize` — Total unique models detected (for UI display) ## Integration with the TUI Application Detection results propagate through [`llmfit-tui/src/tui_app.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_app.rs) and [`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs): ```rust // From tui_app.rs: Application state storage pub struct App { pub docker_mr_available: bool, // ... } pub struct InstalledModels { pub docker_mr: HashSet, pub docker_mr_count: usize, } ``` The UI renders status as "Docker: ✓ (23 models)" or "Docker: ✗" based on these fields, providing immediate visual feedback about containerized model availability. ## Model Pulling and Execution When a user selects a model with a DMR mapping, `llmfit` uses the auto-generated [`docker_models.json`](https://github.com/AlexsJones/llmfit/blob/main/docker_models.json) catalog: ```rust // Pull workflow using docker_mr_pull_tag() if let Some(tag) = docker_mr_pull_tag("bartowski/Llama-3.1-8B-Instruct-GGUF") { match docker_provider.start_pull(&tag) { Ok(handle) => { // Async progress monitoring while let Ok(event) = handle.receiver.recv() { match event { PullEvent::Progress { status, .. } => { println!("{}", status); } PullEvent::Done => { println!("Pull complete"); break; } PullEvent::Error(err) => { eprintln!("Error: {}", err); break; } } } } Err(e) => eprintln!("Failed to start pull: {}", e), } } ``` The `docker_mr_pull_tag()` function maps HuggingFace-style model IDs to DMR-compatible tags using the scraped catalog in [`llmfit-core/data/docker_models.json`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/data/docker_models.json), maintained by [`scripts/scrape_docker_models.py`](https://github.com/AlexsJones/llmfit/blob/main/scripts/scrape_docker_models.py). ## Key Source Files | File | Lines of Interest | Responsibility | |------|-------------------|--------------| | [`llmfit-core/src/providers.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/providers.rs) | 1016-1040, 1658-1671, 1677-1686, 1811-1822, 1885-1892 | Full DMR provider implementation including detection, normalization, and enumeration | | [`llmfit-core/data/docker_models.json`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/data/docker_models.json) | — | Auto-generated model ID to DMR tag mappings | | [`scripts/scrape_docker_models.py`](https://github.com/AlexsJones/llmfit/blob/main/scripts/scrape_docker_models.py) | — | Catalog generation from upstream DMR API | | [`llmfit-tui/src/tui_app.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_app.rs) | 195-197 | Application state storage for detection results | | [`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs) | — | Status rendering in terminal interface | ## Summary - **Docker Desktop verification** runs first through `docker_desktop_installed()` and Linux-specific socket checks to prevent timeouts. - **Configurable connection** via `DOCKER_MODEL_RUNNER_HOST` with `normalize_docker_mr_host()` handling URL normalization. - **Single API probe** to `/v1/models` with early exit on Linux when Docker Desktop is not running. - **Identifier normalization** produces multiple variant forms (full, short, quantization-stripped) for flexible matching. - **Async pull support** through `start_pull()` with progress events for TUI integration. ## Frequently Asked Questions ### Why does llmfit check for Docker Desktop before probing the API? The check prevents an 800 ms HTTP timeout on Linux systems where Docker Desktop is installed but not running. The `is_docker_desktop_running()` function verifies the Unix socket exists before attempting network operations, ensuring the TUI remains responsive. ### Can I use llmfit with Docker Model Runner on a remote host? Yes. Set the `DOCKER_MODEL_RUNNER_HOST` environment variable to your remote DMR endpoint before starting `llmfit`. The `normalize_docker_mr_host()` function handles protocol and trailing slash normalization automatically. ### How does llmfit handle models with different quantization formats? During detection, `DockerModelRunnerProvider::detect_with_installed()` strips quantization tags (e.g., `-Q4_K_M`) from model IDs, storing base names like `llama3.1:8b`. This enables matching user requests against available models regardless of specific quantization variant. ### Where does the model-to-Docker tag mapping come from? The mapping lives in [`llmfit-core/data/docker_models.json`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/data/docker_models.json), auto-generated by [`scripts/scrape_docker_models.py`](https://github.com/AlexsJones/llmfit/blob/main/scripts/scrape_docker_models.py) which queries Docker's official Model Runner API. The `docker_mr_pull_tag()` function performs lookups against this catalog to translate user-facing model IDs to DMR pull commands.