How llmfit Detects Models from Docker Containers: Architecture and Implementation

llmfit uses the Docker Model Runner (DMR) provider to detect containerized models through three steps: verifying Docker Desktop installation, creating a configurable DMR client, and probing the /v1/models endpoint to enumerate installed engines.

The llmfit project implements sophisticated Docker container model detection as part of its multi-provider AI model management system. This article examines how the DockerModelRunnerProvider in llmfit-core/src/providers.rs discovers, enumerates, and prepares Docker-based models for inference, with full source code transparency from the AlexsJones/llmfit repository.

The Docker Model Runner Provider Architecture

llmfit abstracts model detection through the ModelProvider trait, with DockerModelRunnerProvider as its Docker-specific implementation. This design enables pluggable detection for local, remote, and containerized inference engines through a unified interface.

The detection workflow separates platform-specific concerns: Docker Desktop presence checks run first to avoid expensive network timeouts, followed by API probing only when the infrastructure is confirmed available.

Step 1: Verify Docker Desktop Installation

Before attempting any network operations, llmfit confirms Docker Desktop is installed and running through platform-aware checks.

Cross-Platform Installation Detection

The function docker_desktop_installed() in llmfit-core/src/providers.rs scans predefined installation candidates returned by docker_desktop_install_candidates():

• Windows: C:\Program Files\Docker\Docker
• macOS: /Applications/Docker.app
• Linux: /opt/docker-desktop, /usr/bin/docker-desktop
• User directory: ~/.docker/desktop/builds

// From providers.rs: docker_desktop_install_candidates()
// Returns platform-specific paths to check

fn docker_desktop_install_candidates() -> Vec<PathBuf> {
    #[cfg(target_os = "windows")]
    {
        vec![
            PathBuf::from(r"C:\Program Files\Docker\Docker"),
            // Additional Windows paths...
        ]
    }
    #[cfg(target_os = "macos")]
    {
        vec![
            PathBuf::from("/Applications/Docker.app"),
        ]
    }
    #[cfg(target_os = "linux")]
    {
        vec![
            PathBuf::from("/opt/docker-desktop"),
            PathBuf::from("/usr/bin/docker-desktop"),
            home_dir().map(|h| h.join(".docker/desktop")).unwrap_or_default(),
        ]
    }
}

Linux Runtime Verification

On Linux, is_docker_desktop_running() provides an additional guard by checking for the Docker Desktop Unix socket at /run/docker-desktop/docker.sock or ~/.docker/desktop/docker.sock. This check prevents the 800 ms HTTP timeout that would occur if Docker Desktop is installed but not running.

Step 2: Create the DMR Client

With Docker Desktop confirmed, DockerModelRunnerProvider::new() constructs the provider with configurable connection parameters.

Configurable Base URL

The provider defaults to http://localhost:12434 (the standard DMR endpoint) but respects the DOCKER_MODEL_RUNNER_HOST environment variable for custom deployments:

// Create the provider with automatic environment detection
let docker_provider = DockerModelRunnerProvider::new();

// Or with explicit configuration
std::env::set_var("DOCKER_MODEL_RUNNER_HOST", "http://docker-host:12434");
let custom_provider = DockerModelRunnerProvider::new();

The normalize_docker_mr_host() function in llmfit-core/src/providers.rs handles URL normalization, ensuring consistent formatting regardless of trailing slashes or protocol specifications.

Step 3: Probe the DMR API for Installed Models

The detect_with_installed() method performs the actual model enumeration through a single HTTP GET request to {base_url}/v1/models.

Early Exit for Unavailable Infrastructure

On Linux systems, if is_docker_desktop_running() returns false, the method returns early with available = false, avoiding the network timeout entirely. This optimization is critical for TUI responsiveness.

Response Processing and Normalization

Successful responses deserialize into this structure:

#[derive(Deserialize)]
struct DockerModelList {
    data: Vec<DockerEngine>,
}

#[derive(Deserialize)]
struct DockerEngine {
    id: String,           // e.g., "llama3.1:8B-Q4_K_M"
    // Additional metadata...
}

For each engine, llmfit generates multiple identifier variants stored in a HashSet<String>:

Variant	Example	Purpose
Full ID (lowercased)	llama3.1:8b-q4_k_m	Exact matching
Short name	8b-q4_k_m	User-friendly display
Base name (quantization stripped)	llama3.1:8b	Fuzzy matching across formats

// Detection and enumeration example
let (available, models, count) = docker_provider.detect_with_installed();

if available {
    println!("Docker Model Runner: {} models available", count);
    // models contains all normalized identifiers
    for identifier in &models {
        println!("  - {}", identifier);
    }
}

Return Value Structure

The method returns a three-element tuple used throughout the application:

• available: bool — Whether DMR is reachable and responding
• set: HashSet<String> — All normalized model identifiers for membership testing
• count: usize — Total unique models detected (for UI display)

Integration with the TUI Application

Detection results propagate through llmfit-tui/src/tui_app.rs and llmfit-tui/src/tui_ui.rs:

// From tui_app.rs: Application state storage
pub struct App {
    pub docker_mr_available: bool,
    // ...
}

pub struct InstalledModels {
    pub docker_mr: HashSet<String>,
    pub docker_mr_count: usize,
}

The UI renders status as "Docker: ✓ (23 models)" or "Docker: ✗" based on these fields, providing immediate visual feedback about containerized model availability.

Model Pulling and Execution

When a user selects a model with a DMR mapping, llmfit uses the auto-generated docker_models.json catalog:

// Pull workflow using docker_mr_pull_tag()
if let Some(tag) = docker_mr_pull_tag("bartowski/Llama-3.1-8B-Instruct-GGUF") {
    match docker_provider.start_pull(&tag) {
        Ok(handle) => {
            // Async progress monitoring
            while let Ok(event) = handle.receiver.recv() {
                match event {
                    PullEvent::Progress { status, .. } => {
                        println!("{}", status);
                    }
                    PullEvent::Done => {
                        println!("Pull complete");
                        break;
                    }
                    PullEvent::Error(err) => {
                        eprintln!("Error: {}", err);
                        break;
                    }
                }
            }
        }
        Err(e) => eprintln!("Failed to start pull: {}", e),
    }
}

The docker_mr_pull_tag() function maps HuggingFace-style model IDs to DMR-compatible tags using the scraped catalog in llmfit-core/data/docker_models.json, maintained by scripts/scrape_docker_models.py.

Key Source Files

File	Lines of Interest	Responsibility
llmfit-core/src/providers.rs	1016-1040, 1658-1671, 1677-1686, 1811-1822, 1885-1892	Full DMR provider implementation including detection, normalization, and enumeration
llmfit-core/data/docker_models.json	—	Auto-generated model ID to DMR tag mappings
scripts/scrape_docker_models.py	—	Catalog generation from upstream DMR API
llmfit-tui/src/tui_app.rs	195-197	Application state storage for detection results
llmfit-tui/src/tui_ui.rs	—	Status rendering in terminal interface

Summary

• Docker Desktop verification runs first through docker_desktop_installed() and Linux-specific socket checks to prevent timeouts.
• Configurable connection via DOCKER_MODEL_RUNNER_HOST with normalize_docker_mr_host() handling URL normalization.
• Single API probe to /v1/models with early exit on Linux when Docker Desktop is not running.
• Identifier normalization produces multiple variant forms (full, short, quantization-stripped) for flexible matching.
• Async pull support through start_pull() with progress events for TUI integration.

Frequently Asked Questions

Why does llmfit check for Docker Desktop before probing the API?

The check prevents an 800 ms HTTP timeout on Linux systems where Docker Desktop is installed but not running. The is_docker_desktop_running() function verifies the Unix socket exists before attempting network operations, ensuring the TUI remains responsive.

Can I use llmfit with Docker Model Runner on a remote host?

Yes. Set the DOCKER_MODEL_RUNNER_HOST environment variable to your remote DMR endpoint before starting llmfit. The normalize_docker_mr_host() function handles protocol and trailing slash normalization automatically.

How does llmfit handle models with different quantization formats?

During detection, DockerModelRunnerProvider::detect_with_installed() strips quantization tags (e.g., -Q4_K_M) from model IDs, storing base names like llama3.1:8b. This enables matching user requests against available models regardless of specific quantization variant.

Where does the model-to-Docker tag mapping come from?

The mapping lives in llmfit-core/data/docker_models.json, auto-generated by scripts/scrape_docker_models.py which queries Docker's official Model Runner API. The docker_mr_pull_tag() function performs lookups against this catalog to translate user-facing model IDs to DMR pull commands.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}