What Are the Four Dimensions llmfit Uses to Score Models?

llmfit evaluates Large Language Models using four independent dimensions—Quality, Speed, Fit, and Context—which are combined via configurable weights to produce a composite 0-100 score.

The open-source Rust project AlexsJones/llmfit provides an intelligent model selection framework that analyzes hardware capabilities against model requirements. Its scoring system moves beyond simple benchmarks to deliver actionable recommendations through a multi-dimensional assessment encoded in the ScoreComponents struct.

The Four Scoring Dimensions of llmfit

Each dimension represents a critical constraint in production LLM deployments. These values are computed during the ModelFit::analyze method and stored in the ScoreComponents struct defined in llmfit-core/src/fit.rs.

Quality

Quality measures how well the model’s output matches expected results, encompassing accuracy, factuality, and instruction-following capability. This dimension answers whether the model can actually perform the requested task correctly, independent of performance metrics. In the source code, this value is accessed via ScoreComponents::quality at line 1982.

Speed

Speed quantifies the throughput the model can deliver on the detected hardware, measured in tokens-per-second. This dimension accounts for GPU compute capability, memory bandwidth, and quantization levels to predict real-world inference latency. The field ScoreComponents::speed (line 1987) captures this metric during the hardware profiling phase.

Fit

Fit indicates how comfortably the model fits into available memory resources, including GPU VRAM, system RAM, or unified memory architectures like Apple Silicon. A high Fit score ensures the model loads without fragmentation and leaves headroom for context growth. This value resides in ScoreComponents::fit at line 1992.

Context

Context evaluates the amount of context (token window) the model can effectively use without overwhelming memory resources. Unlike raw context length advertised by model cards, this dimension reflects usable context given current hardware constraints. The ScoreComponents::context field at line 1997 stores this calculation.

How llmfit Calculates the Composite Score

The four dimensions are not averaged equally. Instead, llmfit applies configurable weighting tables through the ScoringWeights struct to prioritize different dimensions based on use-case requirements defined in CalcConfig.

During the analysis pipeline, the weighted_score calculation combines these four components into an overall score ranging from 0 to 100. This composite appears throughout the CLI and TUI interfaces with color-coded thresholds for immediate interpretation:

• Green indicates a strong match (score ≥ 70)
• Yellow signals acceptable performance (score ≥ 50)
• Red warns of poor compatibility (score < 50)

The weighting system allows profiles for General usage, Coding, Reasoning, or custom workloads to emphasize different constraints—for example, prioritizing Speed over Context for real-time applications.

Accessing Score Components in Code

Developers can inspect individual dimensions by destructuring the ScoreComponents struct after running ModelFit::analyze:

use llmfit_core::fit::{ModelFit, ScoreComponents};

// Assume `fit` is a `ModelFit` obtained from the analysis pipeline.
let ScoreComponents {
    quality,
    speed,
    fit,
    context,
} = fit.score_components;

// Example: printing the four dimensions
println!("Quality: {:.1}", quality);
println!("Speed:   {:.1}", speed);
println!("Fit:     {:.1}", fit);
println!("Context: {:.1}", context);

// The composite score (0-100) is also available:
println!("Overall score: {:.1}", fit.score);

For interface rendering, the TUI layer in llmfit-tui/src/tui_ui.rs displays these components as parallel data cells:

// In the TUI rendering code – showing the four bars side-by-side
// (excerpt from `llmfit-tui/src/tui_ui.rs`)
ui_cells = vec![
    Cell::from(format!("{:.0}", fit.score_components.quality)),
    Cell::from(format!("{:.0}", fit.score_components.speed)),
    Cell::from(format!("{:.0}", fit.score_components.fit)),
    Cell::from(format!("{:.0}", fit.score_components.context)),
];

The API layer serializes these values in llmfit-tui/src/serve_shared.rs, enabling external tools to consume the four-dimensional assessment for automated model selection pipelines.

Summary

• llmfit scores models using four dimensions: Quality, Speed, Fit, and Context, defined in the ScoreComponents struct in llmfit-core/src/fit.rs.
• The composite score (0-100) is computed via ModelFit::analyze using configurable ScoringWeights tailored to specific use-cases through CalcConfig.
• Color-coded thresholds (green ≥ 70, yellow ≥ 50, red < 50) provide immediate visual feedback on model suitability.
• These components are accessible programmatically, rendered in the TUI, and serialized through the API for integration with external tooling.

Frequently Asked Questions

How does llmfit determine the weighting for each scoring dimension?

llmfit uses the ScoringWeights struct combined with CalcConfig profiles to adjust priority levels per workload. For example, a "Coding" profile might weight Quality and Context higher than Speed, while a "Real-time" profile maximizes Speed and Fit at the expense of Context depth.

Can I view the individual dimension scores separately from the composite score?

Yes. The ScoreComponents struct exposes each dimension as a public field (quality, speed, fit, context) on the ModelFit instance. Both the CLI and TUI interfaces display these four values alongside the composite 0-100 score.

What does a red overall score indicate in llmfit?

A red score (below 50) indicates that the model likely fails to meet minimum requirements across the weighted dimensions. According to the source code in llmfit-core/src/fit.rs, this threshold suggests the model either does not fit in available memory, lacks sufficient context window, or delivers inadequate quality for the specified use-case.

Where is the scoring logic implemented in the llmfit codebase?

The core scoring definitions reside in llmfit-core/src/fit.rs, containing the ScoreComponents struct, weighted_score calculation, and dimension-specific logic. The TUI visualization layer is found in llmfit-tui/src/tui_ui.rs, while API serialization occurs in llmfit-tui/src/serve_shared.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}