LLMFIT Default Scoring Weights Explained: General, Coding, Reasoning, Chat, Multimodal

The default scoring weights for LLMFIT use cases range from 0.10 to 0.55 across four dimensions—quality, speed, fit, and context—with Reasoning prioritizing quality at 0.55 and Chat emphasizing speed at 0.35.

LLMFIT, written in Rust and developed by AlexsJones, is an open-source tool that evaluates how well an LLM fits your hardware configuration. The scoring weights determine how much each performance dimension contributes to the final 0-100 composite score, allowing you to rank models for specific workloads.

Where Default Weights Are Defined

The default scoring weights are implemented in ScoringWeights::default inside llmfit-core/src/fit.rs. The weights are stored as a [[f64; 4]; 6] matrix, where each inner array corresponds to a UseCase variant in enum order.

The four values per use case represent:

1. Quality weight – model accuracy and output quality
2. Speed weight – inference latency and throughput
3. Fit weight – hardware compatibility and resource utilization
4. Context weight – context window suitability

Default Scoring Weights by Use Case

Use Case	Quality	Speed	Fit	Context
General	0.45	0.30	0.15	0.10
Coding	0.50	0.20	0.15	0.15
Reasoning	0.55	0.15	0.15	0.15
Chat	0.40	0.35	0.15	0.10
Multimodal	0.50	0.20	0.15	0.15

Note: The matrix also includes Embedding at [0.30, 0.40, 0.20, 0.10] as the sixth index, defined in the UseCase enum in llmfit-core/src/models.rs.

How Weights Calculate the Final Score

The weighted_score function in llmfit-core/src/fit.rs (lines 86-92) applies these weights using the following formula:

let (wq, ws, wf, wc) = config.scoring_weights.get(use_case);
let score = (quality * wq + speed * ws + fit * wf + context * wc) / (wq + ws + wf + wc);

This normalization ensures the final score remains comparable even when weight distributions vary significantly between use cases.

Retrieving Weights Programmatically

Access the default weights for any use case through the CalcConfig and ScoringWeights API:

use llmfit_core::fit::{CalcConfig, ScoringWeights, UseCase};

fn main() {
    // Create a config with the built-in defaults
    let cfg = CalcConfig::default();

    // Retrieve the weight tuple for the Coding use-case
    let (quality_w, speed_w, fit_w, context_w) =
        cfg.scoring_weights.get(UseCase::Coding);

    println!(
        "Coding weights → quality: {}, speed: {}, fit: {}, context: {}",
        quality_w, speed_w, fit_w, context_w
    );
    // Output: Coding weights → quality: 0.5, speed: 0.2, fit: 0.15, context: 0.15
}

Customizing Scoring Weights

Override defaults by modifying the weights array directly in your CalcConfig:

let mut cfg = CalcConfig::default();
cfg.scoring_weights.weights[UseCase::Coding as usize] = [0.30, 0.50, 0.10, 0.10];

This example prioritizes speed over quality for coding workloads—useful when latency matters more than perfect code generation.

Alternatively, use the TUI's Advanced Configuration panel in llmfit-tui/src/main.rs to adjust weights interactively without writing code.

Design Rationale Behind Defaults

• Reasoning (0.55 quality): Complex problem-solving demands highest accuracy, accepting slower inference
• Chat (0.35 speed): Conversational interfaces require responsive, low-latency interactions
• Coding (0.50 quality, 0.20 speed): Balances correctness with reasonable compile-test cycles
• Multimodal (equal 0.20 speed/context): Handles diverse input types requiring balanced resource allocation
• General (0.45 quality, 0.30 speed): Versatile default for unspecified workloads

Related Source Files

File	Purpose
llmfit-core/src/fit.rs	ScoringWeights struct, default() implementation, weighted_score() calculation
llmfit-core/src/models.rs	UseCase enum defining matrix indices
llmfit-tui/src/main.rs	Interactive weight configuration interface

Summary

• LLMFIT scoring weights are four-dimensional vectors (quality, speed, fit, context) stored as f64 values in llmfit-core/src/fit.rs
• Reasoning maximizes quality weight at 0.55; Chat maximizes speed weight at 0.35
• Weights normalize to produce a 0-100 composite score via weighted_score()
• Override defaults programmatically through CalcConfig or via the TUI configuration panel
• The UseCase enum in models.rs determines matrix indexing order

Frequently Asked Questions

Can I set custom weights for individual models rather than use cases?

No. LLMFIT applies scoring weights at the use-case level, not per-model. Configure your CalcConfig with weights tailored to your workflow, then run evaluation against multiple models. The same weight vector ranks all models for that use case.

Why does the Reasoning use case have the lowest speed weight?

Reasoning tasks—mathematical proofs, logical deduction, strategic planning—prioritize correctness over latency. The 0.15 speed weight reflects that users typically accept slower inference for accurate, well-reasoned outputs. This is hardcoded in ScoringWeights::default at lines 86-92 of fit.rs.

Are these weights used in the CLI or just the TUI?

Both interfaces use the same underlying CalcConfig system. The TUI exposes weight adjustment through its Advanced Configuration panel, while the CLI and programmatic API require manual CalcConfig construction. All paths ultimately call weighted_score() in llmfit-core/src/fit.rs.

How do I verify which weights are active during a scoring run?

Enable debug logging or inspect your CalcConfig before evaluation. The config's scoring_weights field contains the active [[f64; 4]; 6] matrix. Access your specific use case weights with .get(UseCase::Variant), which returns a (f64, f64, f64, f64) tuple.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}