How to Run llmfit in Docker Interactively

Run docker run --rm -it ghcr.io/alexsjones/llmfit --tui to launch the fully-featured terminal UI inside a container, with the same hardware detection and model fitting capabilities as a native build.

The llmfit project distributes a ready-to-run container on GitHub Container Registry that enables you to run llmfit in Docker interactively without local compilation. When started with an attached terminal (-it) and the global --tui flag, the container executes the ratatui-based interface defined in llmfit-tui/src/main.rs, accessing host CPU, RAM, and GPU information directly through the shared kernel.

Prerequisites

• Docker Engine (or Podman) installed locally
• Network access to ghcr.io/alexsjones/llmfit
• Optional: NVIDIA Container Toolkit for GPU passthrough

Basic Interactive Docker Command

To start the interactive TUI, allocate a pseudo-TTY and keep stdin open with the -it flags:

docker run --rm -it ghcr.io/alexsjones/llmfit --tui

This command launches the application’s entry point in llmfit-tui/src/main.rs, which parses the --tui flag and initializes the terminal interface immediately upon container startup.

Enabling GPU Access in the Container

For accurate VRAM detection and hardware-accelerated model fitting analysis, pass your NVIDIA GPUs into the container using the --gpus flag:

docker run --rm -it --gpus all ghcr.io/alexsjones/llmfit --tui

The --gpus all parameter allows the sysinfo crate and nvidia-smi helpers inside llmfit-core/src/hardware.rs to query device capabilities, enabling the fit analysis engine in llmfit-core/src/fit.rs to calculate compatibility levels (Perfect, Good, etc.) against the embedded model catalog.

Using Podman Instead of Docker

The same container image works identically under Podman without modification:

podman run --rm -it ghcr.io/alexsjones/llmfit --tui

Podman users can also add --gpus all when running rootless containers with the appropriate GPU delegation configuration.

Architecture Behind Interactive Docker Support

Entry Point and TUI Initialization

In llmfit-tui/src/main.rs, the application parses command-line arguments and launches the interactive interface when the --tui flag is present. Because this is a compiled Rust binary running inside the container, execution behavior matches native builds exactly.

Host Hardware Detection

The hardware detection module at llmfit-core/src/hardware.rs leverages the fact that containers share the host kernel. It reads CPU and RAM statistics via the sysinfo crate and executes nvidia-smi or rocm-smi helpers to gather GPU specifications. This allows the UI to display real-time host hardware information even from within the isolated container environment.

Embedded Model Database

The application embeds the HuggingFace model catalog directly into the binary using include_str!("../data/hf_models.json") in llmfit-core/src/models.rs. This eliminates external file I/O dependencies, ensuring the TUI functions completely self-contained without requiring mounted volumes or network access to load model data.

Fit Analysis Engine

Located in llmfit-core/src/fit.rs, the fit calculation logic consumes the detected hardware specs and compares them against embedded model requirements to generate compatibility ratings. This operates identically inside Docker because it relies only on system calls available through the container runtime.

Running Non-Interactive Commands

You can also execute one-off commands without the TUI. For example, to output a JSON list of recommended models based on detected hardware:

docker run --rm ghcr.io/alexsjones/llmfit recommend --json

Summary

• Run interactively using docker run --rm -it ghcr.io/alexsjones/llmfit --tui to launch the ratatui-based terminal UI
• Enable GPU detection by adding --gpus all for accurate VRAM and hardware analysis via llmfit-core/src/hardware.rs
• Use Podman interchangeably with the same command syntax and image reference
• No volume mounts required because the model database is embedded at compile time via include_str!() in llmfit-core/src/models.rs
• Identical functionality to native builds since the container runs the same Rust binary with direct kernel access for hardware queries

Frequently Asked Questions

Does the Docker container require GPU access to run the TUI?

No, GPU access is optional. The TUI functions fully without the --gpus flag, though hardware detection will be limited to CPU and RAM statistics. Add --gpus all only when you need accurate GPU VRAM calculations for model fitting analysis.

How does llmfit detect host hardware from inside a Docker container?

The container shares the host kernel, allowing llmfit-core/src/hardware.rs to access system information through standard syscalls via the sysinfo crate. GPU detection requires passing device drivers through using Docker's --gpus flag, which exposes nvidia-smi or rocm-smi interfaces to the containerized binary.

Can I use Podman instead of Docker to run llmfit interactively?

Yes, the ghcr.io/alexsjones/llmfit image is fully compatible with Podman. Execute podman run --rm -it ghcr.io/alexsjones/llmfit --tui with identical flags. GPU support requires rootless Podman configuration for device delegation, but the interactive TUI experience remains unchanged.

Is the interactive experience identical between Docker and native builds?

Yes, the experience is identical because the container executes the same compiled binary. The UI initialization in llmfit-tui/src/main.rs, hardware detection logic, and embedded model database in llmfit-core/src/models.rs function exactly as they would in a locally compiled version, with no host-specific library dependencies beyond standard kernel interfaces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}