How to Override Memory Bandwidth or Efficiency Settings in llmfit

You can override memory bandwidth and efficiency settings in llmfit through hardware profiles, the TUI Advanced Configuration panel, environment variables, or CLI flags to customize TPS estimates for your specific hardware.

llmfit automatically detects hardware specifications to estimate tokens-per-second (TPS) performance using roof-line analysis, but you can override memory bandwidth or efficiency settings in llmfit to model custom hardware, unreleased GPUs, or alternative efficiency assumptions. The configuration system centers on the CalcConfig struct defined in llmfit-core/src/fit.rs, which accepts overrides for GPU memory bandwidth, system DDR bandwidth, and kernel efficiency factor through multiple interfaces. These values directly control the throughput estimates calculated by resolve_gpu_bandwidth and the TPS conversion logic in the core engine.

Override GPU Memory Bandwidth

The GPU memory bandwidth value drives the decode-throughput estimate in roof-line calculations. By default, resolve_gpu_bandwidth queries the system hardware, but you can force a specific value through three mechanisms that write to CalcConfig::gpu_bandwidth_gbps_override.

Via Hardware Profile

Create a JSON hardware profile containing the gpu_memory_bandwidth_gbps field. The parser in llmfit-core/src/hwprofile.rs (lines 330-339) validates this value via HardwareProfile::validate and copies it into CalcConfig, overriding automatic detection.

{
  "hardware": {
    "gpu_memory_bandwidth_gbps": 777.0
  }
}

Run llmfit with the profile:

llmfit --profile ./my_profile.json fit --perfect

Via TUI Advanced Configuration

Start the TUI and press A to open the Advanced Configuration panel. The input field adv_config_gpu_bandwidth_input defined in llmfit-tui/src/tui_app.rs captures your value, while llmfit-tui/src/tui_ui.rs renders the editable interface. Changes write directly to CalcConfig before any fit calculation executes.

Via Environment Variable

For rapid testing without file edits, export LLMFIT_GPU_BANDWIDTH_GBPS. This fallback is read in llmfit-core/src/hwprofile.rs (lines 31-34) when no hardware profile is supplied.

export LLMFIT_GPU_BANDWIDTH_GBPS=777
llmfit fit --perfect

Override System DDR Bandwidth

The DDR bandwidth parameter controls throughput estimates for MoE-offload and CPU-offload paths, stored in CalcConfig::ddr_bandwidth_gbps and accessed by the ddr_bandwidth_gbps() function.

Via Hardware Profile

Include the ddr_bandwidth_gbps key in your hardware profile JSON. This value is parsed by the same validation logic in llmfit-core/src/hwprofile.rs and injected into the calculation config.

{
  "hardware": {
    "ddr_bandwidth_gbps": 250.0
  }
}

Via TUI Advanced Configuration

In the TUI Advanced Configuration panel (press A), edit the DDR bandwidth field. The UI updates CalcConfig::ddr_bandwidth_gbps immediately upon confirmation.

Via Environment Variable

Set LLMFIT_DDR_BANDWIDTH before invocation to override the detected system memory bandwidth.

export LLMFIT_DDR_BANDWIDTH=250
llmfit fit --perfect

Override the Efficiency Factor

The efficiency factor represents the fraction of peak memory bandwidth that kernels can actually sustain, defaulting to 0.55 as defined in default_efficiency() at line 65 of llmfit-core/src/fit.rs. This scalar directly scales the final TPS estimate in the roof-line model.

Via Hardware Profile

Add the efficiency field to your hardware profile to change the default 0.55 value.

{
  "hardware": {
    "efficiency": 0.80
  }
}

Via TUI Advanced Configuration

Navigate to the Efficiency field in the TUI Advanced Configuration panel and enter a decimal value (e.g., 0.80 for 80% efficiency).

Via CLI Flag

Pass --efficiency followed by a fractional value. This argument is defined in llmfit-tui/src/main.rs using clap and stored directly in CalcConfig::efficiency.

llmfit --efficiency 0.80 fit --perfect

Summary

• GPU memory bandwidth overrides flow through CalcConfig::gpu_bandwidth_gbps_override via hardware profiles (gpu_memory_bandwidth_gbps), the TUI Advanced Configuration panel, or the LLMFIT_GPU_BANDWIDTH_GBPS environment variable.
• System DDR bandwidth is controlled via CalcConfig::ddr_bandwidth_gbps using profile JSON (ddr_bandwidth_gbps), TUI inputs, or the LLMFIT_DDR_BANDWIDTH environment variable.
• Efficiency factor defaults to 0.55 in llmfit-core/src/fit.rs but can be changed through hardware profiles, the TUI, or the --efficiency CLI flag.
• All overrides ultimately populate CalcConfig fields that drive the resolve_gpu_bandwidth logic and TPS conversion calculations.

Frequently Asked Questions

What is the default efficiency factor in llmfit?

The default efficiency factor is 0.55 (55%), defined in the default_efficiency() function at line 65 of llmfit-core/src/fit.rs. This value represents the assumed fraction of peak memory bandwidth that GPU kernels can sustain during actual inference workloads.

How do I quickly test different bandwidth values without creating a profile file?

Use the environment variables LLMFIT_GPU_BANDWIDTH_GBPS and LLMFIT_DDR_BANDWIDTH. These are parsed in llmfit-core/src/hwprofile.rs (lines 31-34) as fallbacks when no hardware profile is provided, allowing immediate testing of alternative bandwidth scenarios.

Which source file handles the hardware profile parsing?

Hardware profile parsing and validation occur in llmfit-core/src/hwprofile.rs. This file contains the HardwareProfile::validate method (around lines 330-339) that extracts gpu_memory_bandwidth_gbps, ddr_bandwidth_gbps, and efficiency from JSON profiles and maps them to CalcConfig fields.

Can I override settings through the TUI without restarting llmfit?

Yes. The TUI Advanced Configuration panel (accessed by pressing A) updates CalcConfig fields in real-time through the adv_config_*_input state handlers in llmfit-tui/src/tui_app.rs before any fit or plan calculation runs, so you do not need to restart the application to apply new bandwidth or efficiency values.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}