How to Configure Custom Scoring Weights for Different llmfit Use Cases

llmfit supports custom scoring weights through the ScoringWeights struct in fit.rs, allowing you to adjust the relative importance of Quality, Speed, Fit, and Context across six built-in use cases.

The llmfit framework evaluates every LLM across four independent dimensions—Quality, Speed, Fit, and Context—then combines them into a single composite score. The ScoringWeights struct in llmfit-core/src/fit.rs controls how heavily each dimension contributes per use case, making it possible to tune rankings for specific workloads like coding, reasoning, or chat applications.

Understanding the Default Scoring Weights in llmfit

The default weight matrix is hardcoded in the Default implementation of ScoringWeights (llmfit-core/src/fit.rs, lines 76-98). Each of the six use cases receives a four-element array representing the relative weights for Quality, Speed, Fit, and Context:

Use Case	Quality	Speed	Fit	Context
General	0.25	0.25	0.25	0.25
Coding	0.20	0.30	0.30	0.20
Reasoning	0.15	0.20	0.25	0.40
Chat	0.10	0.35	0.30	0.25
Multimodal	varies	varies	varies	varies
Embedding	varies	varies	varies	varies

The composite score calculation occurs in the weighted_score function (lines 1729-1732):

let (wq, ws, wf, wc) = config.scoring_weights.get(use_case);
let composite = (quality * wq + speed * ws + fit * wf + context * wc) / (wq + ws + wf + wc);

Method 1: Programmatic Configuration in Rust

For direct library usage, construct a custom ScoringWeights instance and inject it into your Config before calling the analysis API:

use llmfit_core::{
    fit::{ScoringWeights, UseCase},
    Config,
};

fn custom_config() -> Config {
    // 6×4 matrix: six use cases, four dimensions each
    let my_weights = ScoringWeights::new([
        // General: balanced
        [0.30, 0.20, 0.30, 0.20],
        // Coding: prioritize speed and fit
        [0.15, 0.40, 0.30, 0.15],
        // Reasoning: prioritize context window
        [0.10, 0.15, 0.25, 0.50],
        // Chat: maximize speed for responsiveness
        [0.05, 0.45, 0.35, 0.15],
        // Multimodal
        [0.12, 0.25, 0.30, 0.33],
        // Embedding: maximize context for document processing
        [0.25, 0.10, 0.15, 0.50],
    ]);

    let mut cfg = Config::default();
    cfg.scoring_weights = my_weights;
    cfg
}

The ScoringWeights::new constructor and UseCase enum are both defined in llmfit-core/src/fit.rs.

Method 2: TUI Advanced Configuration Panel

The interactive TUI provides real-time weight editing without code changes:

1. Launch the TUI: llmfit
2. Press A to open the Advanced Configuration panel
3. Navigate to the desired use case row
4. Edit the four weight columns (Quality, Speed, Fit, Context)
5. Press Enter to apply—the next scoring refresh uses your values

This writes modified weights directly into the runtime configuration object. Documentation for this interface appears in docs/tui.md under the "Advanced Configuration" section.

Method 3: CLI via Custom Configuration File

Pass a TOML or YAML file matching the internal Config structure:

# speed_focused.toml

scoring_weights = [
  [0.15, 0.45, 0.30, 0.10], # General

  [0.10, 0.50, 0.30, 0.10], # Coding

  [0.05, 0.25, 0.25, 0.45], # Reasoning

  [0.05, 0.55, 0.30, 0.10], # Chat

  [0.10, 0.30, 0.30, 0.30], # Multimodal

  [0.20, 0.15, 0.15, 0.50], # Embedding

]

Execute with:

llmfit --config speed_focused.toml fit --top 10

The CLI argument parsing in llmfit-tui/src/main.rs deserializes this file and forwards it to the core library via serde.

Method 4: Python Wrapper Integration

The Python package forwards arguments to the compiled binary, so the configuration file approach works identically:

import subprocess

subprocess.run([
    "llmfit",
    "--config", "my_weights.toml",
    "fit",
    "--top", "10"
])

Alternatively, generate temporary config files programmatically to match dynamic weighting requirements.

Choosing the Right Weight Distribution

Different operational priorities demand different scoring weight configurations:

• High-throughput chat services: Increase Speed weight (0.40-0.55) to minimize latency
• Accuracy-critical reasoning pipelines: Increase Quality and Context weights for complex problem-solving
• Code completion tools: Balance Speed and Fit weights for responsive, relevant suggestions
• Document embedding clusters: Maximize Context weight (0.50+) for processing long documents

Summary

• Source of truth: ScoringWeights struct in llmfit-core/src/fit.rs (lines 76-98)
• Scoring logic: weighted_score function (lines 1729-1732)
• Use cases: Six variants defined in the UseCase enum
• Configuration methods: Rust API, TUI panel, CLI config file, or Python wrapper
• File format: Nested 6×4 array in TOML/YAML matching ScoringWeights::new signature

Frequently Asked Questions

How does llmfit normalize scoring weights if they don't sum to 1.0?

The weighted_score function divides by the sum of all four weights (wq + ws + wf + wc), so weights automatically normalize regardless of absolute values. You can use raw integers (10, 30, 20, 40) or decimals (0.1, 0.3, 0.2, 0.4) interchangeably.

Can I define custom use cases beyond the six built-in categories?

The UseCase enum in llmfit-core/src/fit.rs currently enumerates six fixed variants. Adding custom use cases requires modifying the source and rebuilding, or using the closest existing category with locally tuned weights. The library architecture treats use cases as array indices, so the matrix dimension is compile-time fixed.

Where does llmfit store weights edited through the TUI?

TUI modifications persist only for the current process lifetime unless explicitly saved. The TUI updates the in-memory Config struct; for permanent changes, export the configuration to a file or modify your startup config. No automatic persistence to disk occurs on edit.

What happens if I provide a malformed weights array in my config file?

serde deserialization fails with a clear error indicating expected versus actual dimensions. The ScoringWeights constructor validates the 6×4 shape, and the CLI in llmfit-tui/src/main.rs surfaces these errors before any scoring begins.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}