# How LLMFIT_DDR_BANDWIDTH Configures MoE Expert Streaming Estimates in llmfit

> Discover how LLMFIT_DDR_BANDWIDTH configures MoE expert streaming estimates in llmfit. Override system RAM calculations for accurate throughput predictions in MoE off-load inference.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: internals
- Published: 2026-08-21

---

**The `LLMFIT_DDR_BANDWIDTH` environment variable overrides system RAM bandwidth calculations to control throughput predictions for Mixture-of-Experts (MoE) off-load inference in the llmfit engine.**

When running large MoE models with limited GPU VRAM, the llmfit tool streams inactive expert weights from system RAM. The `LLMFIT_DDR_BANDWIDTH` setting directly determines how the `AlexsJones/llmfit` repository calculates tokens-per-second (TPS) estimates for these streaming workloads, ensuring predictions match your actual hardware capabilities.

## The Role of RAM Bandwidth in MoE Off-load

Mixture-of-Experts architectures place unique demands on system memory. In the MoE off-load path implemented in [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs), only active experts reside in GPU VRAM while inactive weights remain in system RAM and stream on demand.

This creates a memory-bound bottleneck where **inference speed depends entirely on DDR throughput**. The estimation engine calculates expert loading time using the formula:

```rust
// expert read time = active_gb / ddr_bandwidth_gbps
let expert_read_time = active_gb / ddr_bandwidth_gbps(config);

```

This calculation occurs specifically within `RunMode::MoeOffload` logic (lines 31-41 of *fit.rs*), making the bandwidth value critical for accurate TPS forecasting on MoE models like Mixtral 8x7b.

## Configuration Priority Hierarchy

The `ddr_bandwidth_gbps` function in *llmfit-core/src/fit.rs* resolves the effective bandwidth through a strict priority chain. The system checks these sources in order:

1. **TUI Advanced Config** – The `CalcConfig::ddr_bandwidth_gbps` field set via the graphical interface takes highest precedence.

2. **`LLMFIT_DDR_BANDWIDTH` Environment Variable** – If present, the raw numeric value (in GB/s) overrides hardware detection. The parser validates positive floating-point values:

```rust
if let Some(bw) = std::env::var("LLMFIT_DDR_BANDWIDTH")
    .ok()
    .and_then(|v| v.parse::<f64>().ok())
    .filter(|b| *b > 0.0) { … }

```

This logic appears at lines 1158-1165 of *fit.rs*.

3. **Hardware Measurement** – When no override exists, `hardware::measured_ram_bandwidth_gbps()` probes the actual system capabilities.

4. **Conservative Fallback** – If measurement fails, the engine defaults to **50 GB/s**, representing typical DDR4-3200 dual-channel bandwidth.

## Practical Usage Examples

### Override Bandwidth via Environment Variable

Set the variable before running the fit command to simulate or enforce specific RAM constraints:

```bash

# Force 90 GB/s DDR bandwidth estimate

export LLMFIT_DDR_BANDWIDTH=90
llmfit fit --model mixtral-8x7b

```

The TPS calculation will use 90 GB/s instead of measured or fallback values, reflecting high-end DDR5 or quad-channel configurations.

### Programmatic Access in Rust

For custom tooling that replicates llmfit's logic:

```rust
use std::env;

fn get_ddr_bw() -> f64 {
    env::var("LLMFIT_DDR_BANDWIDTH")
        .ok()
        .and_then(|v| v.parse::<f64>().ok())
        .filter(|b| *b > 0.0)
        .unwrap_or(50.0) // matches llmfit fallback
}
println!("Using DDR bandwidth: {} GB/s", get_ddr_bw());

```

### TUI Advanced Configuration

The graphical interface overrides both environment variables and hardware detection through `CalcConfig`:

```rust
let cfg = CalcConfig {
    ddr_bandwidth_gbps: Some(120.0), // 120 GB/s from UI
    ..Default::default()
};
let fit = ModelFit::analyze_with_config(&model, &system, cfg);
println!("Estimated TPS (MoE offload): {}", fit.estimated_tps);

```

This pattern appears in [`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs) where the Advanced Configuration panel exposes direct bandwidth control.

## Implementation Reference

The bandwidth override mechanism spans three core components:

- **[`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs)** – Contains the `ddr_bandwidth_gbps` resolution logic and MoE-offload TPS estimation. The environment variable parsing occurs at lines 1158-1165.
  
- **[`llmfit-core/src/hardware.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/hardware.rs)** – Provides the `measured_ram_bandwidth_gbps` fallback function when `LLMFIT_DDR_BANDWIDTH` remains unset.

- **[`llmfit-tui/src/tui_ui.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-tui/src/tui_ui.rs)** – Implements the TUI Advanced Configuration panel that writes to `CalcConfig::ddr_bandwidth_gbps`, superseding environment variables.

Additional architectural context appears in [`docs/how-it-works.md`](https://github.com/AlexsJones/llmfit/blob/main/docs/how-it-works.md), which documents the MoE streaming estimation pipeline.

## Summary

- **`LLMFIT_DDR_BANDWIDTH`** overrides system RAM speed detection for MoE throughput estimates, accepting values in GB/s.
- The variable sits at priority level two in the bandwidth resolution chain, below TUI config but above hardware measurement.
- It directly determines `expert_read_time` calculations when `RunMode::MoeOffload` is active, affecting tokens-per-second predictions.
- If omitted, llmfit either measures actual RAM bandwidth or falls back to 50 GB/s (DDR4-3200 dual-channel).
- Setting this variable is essential when targeting high-throughput DDR5 systems or when hardware detection produces inaccurate readings.

## Frequently Asked Questions

### What happens if I don't set LLMFIT_DDR_BANDWIDTH?

If the environment variable is unset and no TUI configuration exists, llmfit calls `hardware::measured_ram_bandwidth_gbps()` to probe your system's actual memory throughput. If measurement fails or returns invalid data, the engine conservatively assumes **50 GB/s**, matching standard DDR4-3200 dual-channel performance. This ensures estimates remain realistic on typical consumer hardware.

### Can I use this variable for non-MoE models?

No. The bandwidth override only affects the estimation path when `RunMode::MoeOffload` is active and the model architecture is detected as a Mixture-of-Experts. Dense models that load entirely into VRAM do not stream weights from system RAM, so `LLMFIT_DDR_BANDWIDTH` has no impact on their TPS calculations.

### How does the TUI Advanced Config interact with the environment variable?

The TUI configuration takes absolute precedence. When `CalcConfig::ddr_bandwidth_gbps` contains `Some(value)`, the `ddr_bandwidth_gbps` function in *fit.rs* returns that value immediately without checking environment variables or hardware measurements. To use `LLMFIT_DDR_BANDWIDTH`, ensure the TUI field is cleared or set to `None`.

### What units should I use for LLMFIT_DDR_BANDWIDTH?

Specify the value in **gigabytes per second (GB/s)** as a floating-point number. For example, use `90` for 90 GB/s or `120.5` for 120.5 GB/s. The parser rejects non-numeric strings and values less than or equal to zero, falling back to hardware detection or the 50 GB/s default if validation fails.