What Is llmfit-core? The Rust Engine Behind Hardware-Aware LLM Recommendations

llmfit-core is the shared Rust library that detects your system's hardware specifications, evaluates large language model compatibility, and generates quantitative "fit" scores to recommend optimal models for your specific CPU, RAM, and GPU configuration.

The llmfit-core crate serves as the computational backbone of the AlexsJones/llmfit repository, providing deterministic hardware analysis and model ranking logic consumed by the CLI, TUI, HTTP API, and Python wrapper interfaces. This library abstracts away the complexity of evaluating LLM compatibility across diverse hardware setups and runtime environments.

Hardware Detection with SystemSpecs

The library gathers comprehensive host information through the SystemSpecs struct implemented in src/hardware.rs. Calling SystemSpecs::detect() performs system introspection to identify available CPU cores, total RAM capacity, installed GPUs, available VRAM, and backend execution capabilities required for runtime selection.

Model Catalog and Database Management

In src/models.rs, the ModelDatabase structure handles ingestion of the embedded hf_models.json catalog alongside custom user-defined models. This module deserializes model metadata—including parameter counts, quantization formats, and context window limits—into a queriable internal format that the fit analysis engine consumes.

Quantitative Fit Analysis

The core recommendation logic resides in src/fit.rs, where the ModelFit struct and its analyze_* methods determine the optimal execution path for each model. For every candidate model, llmfit-core evaluates:

• Runtime selection – Chooses between CUDA, Metal, ROCm, or CPU-only execution
• Quantization strategy – Identifies compatible GGUF variants or MLX formats
• Memory estimation – Calculates RAM and VRAM requirements for loading
• Performance prediction – Estimates tokens-per-second (TPS) throughput
• Composite scoring – Generates a percentage-based utilization score and categorical fit level

Installed Model Detection

The src/analysis.rs module provides the InstalledIndex type, which queries all supported inference providers in parallel to identify locally available models. The InstalledIndex::detect_all() method searches across:

• Ollama
• MLX
• Docker Model Runner
• LM Studio
• vLLM
• RamaLama

This detection prevents redundant download recommendations and adjusts scoring for models already cached on disk.

Benchmark Calibration and Scoring

To improve prediction accuracy, src/analysis.rs implements apply_local_calibration and related helper methods that incorporate user-reported benchmark results and community telemetry. The scoring system in src/fit.rs applies configurable weights—prioritizing quality, speed, hardware fit, or context length—via rank_models_by_fit_opts_* functions to sort results for UI consumption.

How to Use llmfit-core in Rust Applications

The following patterns demonstrate how consumer applications like the llmfit-tui binary interact with the library's public API exposed through src/lib.rs.

Detecting Hardware and Loading the Catalog

use llmfit_core::{hardware::SystemSpecs, models::ModelDatabase};

// Gather system specifications
let specs = SystemSpecs::detect();

// Load the embedded HuggingFace model database
let db = ModelDatabase::new();

Building Model Fit Recommendations

use llmfit_core::{
    analysis::{InstalledIndex, build_model_fits},
    fit::InferenceRuntime,
};

// Detect locally installed models across all providers
let installed = InstalledIndex::detect_all();

// Generate fit analysis for all compatible models
let fits = build_model_fits(
    &db,
    &specs,
    &installed,
    None,  // Optional context-length cap
    None,  // Optional forced runtime override
);

Ranking and Displaying Results

use llmfit_core::fit::rank_models_by_fit;

// Sort by composite fit score
let mut ranked = rank_models_by_fit(fits);
ranked.truncate(10); // Limit to top 10 recommendations

for fit in ranked {
    println!(
        "{} – {} – {:.1}% – {:.1} tok/s",
        fit.model.name,
        fit.fit_text(),
        fit.utilization_pct,
        fit.estimated_tps,
    );
}

Key Source Files and Architecture

Understanding the module structure helps developers extend or debug the library:

• src/lib.rs – Public re-exports and crate entry point exposing ModelFit, FitLevel, RunMode, and SystemSpecs
• src/hardware.rs – System detection implementation for CPU, RAM, GPU, and backend identification
• src/models.rs – Model metadata structures and hf_models.json parsing logic
• src/fit.rs – Core analysis engine, ModelFit struct definitions, and ranking algorithms
• src/analysis.rs – Installed model indexing, calibration logic, and provider aggregation
• src/providers.rs – Provider-specific abstractions used by the InstalledIndex detection system

Integration with Front-End Interfaces

llmfit-core functions as the single source of truth for all user-facing applications in the llmfit ecosystem. The llmfit-tui crate (the interactive terminal binary), the HTTP API server, and the Python wrapper all delegate hardware analysis and model ranking to this library. This centralization ensures consistent recommendations whether users interact via command-line flags, JSON REST endpoints, or programmatic Python imports.

Summary

• llmfit-core provides the deterministic engine that transforms raw hardware specifications into ranked LLM recommendations
• SystemSpecs::detect() in src/hardware.rs gathers CPU, RAM, GPU, and VRAM details required for compatibility checks
• ModelDatabase and ModelFit structs in src/models.rs and src/fit.rs handle catalog ingestion and quantitative scoring
• InstalledIndex::detect_all() queries Ollama, MLX, vLLM, and other providers to identify locally cached models
• The library exposes a unified Rust API through src/lib.rs consumed by CLI, TUI, web, and Python interfaces

Frequently Asked Questions

What is the primary purpose of llmfit-core?

llmfit-core implements the hardware detection, model catalog parsing, and quantitative scoring algorithms required to recommend large language models that will run efficiently on a specific machine. It serves as the shared library backing all llmfit user interfaces.

Which hardware components does llmfit-core detect?

According to the source code in src/hardware.rs, the library detects CPU core count, total system RAM, GPU availability and model, VRAM capacity, and supported compute backends (CUDA, Metal, ROCm) to determine compatible execution runtimes.

How does llmfit-core handle different LLM providers?

The src/providers.rs module defines abstractions for each supported inference engine. The InstalledIndex type in src/analysis.rs queries these providers—including Ollama, MLX, Docker Model Runner, LM Studio, vLLM, and RamaLama—in parallel to build a complete index of locally installed models.

Can I use llmfit-core without the CLI interface?

Yes. While llmfit-core powers the official CLI and TUI binaries, it exports a public Rust API through src/lib.rs that any Rust application can consume. Third-party tools can call SystemSpecs::detect(), ModelDatabase::new(), and build_model_fits() directly to integrate hardware-aware recommendations into their own workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}