# Where Local, Community, and Measured Presets Take Precedence in llmfit Benchmarks

> Discover how llmfit prioritizes local measured benchmarks over community presets for faster, more accurate results. Learn the fitting logic in llmfit-core/src/fit.rs.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: benchmarks
- Published: 2026-09-12

---

**The fitting routine prioritizes local measured benchmarks over community presets, falling back to cached community data only when local measurements are unavailable, as implemented in [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs).**

Understanding **local, community, and measured presets precedence** is essential when estimating model performance in the AlexsJones/llmfit project. The fitting logic residing in [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs) enforces a strict hierarchy that prefers user-generated measurements from `llmfit bench` commands over pre-computed community leaderboard data. This article examines exactly how the precedence logic determines which preset data drives the estimation process.

## Understanding the Confidence Hierarchy

The precedence system relies on the `EstimateConfidence` enum defined in [`fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/fit.rs), which establishes a clear priority order for data sources during model fitting.

### The EstimateConfidence Enum Definition

Located at lines 276 and 287 of [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs), this enum codifies the hierarchy:

```rust
// Simplified representation of the enum hierarchy (fit.rs#L276-L287)
pub enum EstimateConfidence {
    Measured,         // Local benchmark data (highest priority)
    MeasuredCommunity, // Cached community leaderboard
    Formula,          // Pure theoretical calculation
}

```

`EstimateConfidence::Measured` represents data from a user-run `llmfit bench` command, while `MeasuredCommunity` indicates fallback to cached leaderboard entries. `Formula` serves as the lowest-confidence fallback when no empirical data exists.

## How Local Benchmarks Take Precedence

The `ModelFit::analyze` method and its helper `estimate_basis` implement the primary check for local measurements. When local throughput data exists, the system bypasses community calibration entirely.

### The Local Measurement Check (Lines 3049-3068)

After constructing a fit, the code explicitly checks for local benchmark availability before considering any community data:

```rust
// Inside ModelFit::analyze – precedence decision logic (fit.rs#L3049-L3068)
if let Some(local_meas) = self.measured_tps {
    // Local benchmark exists → use directly with highest confidence
    self.estimate_confidence = EstimateConfidence::Measured;
} else {
    // No local measurement → proceed to community fallback
    self.estimate_confidence = EstimateConfidence::MeasuredCommunity;
}

```

This block ensures that **local measured benchmarks dominate** the fitting process whenever present. If `self.measured_tps` contains valid throughput data, the routine immediately assigns `EstimateConfidence::Measured` and skips the community calibration loop.

## Community Calibration as Fallback Mechanism

Only when `measured_tps` remains unset does the estimator invoke the community calibration loop. This secondary process iterates through cached preset labels to load pre-computed leaderboard data from disk.

### The Community Calibration Loop (Lines 3145-3149)

The fallback logic appears in the latter sections of [`fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/fit.rs), executing exclusively when local measurements are absent:

```rust
// Community calibration executes only when local measurement is absent (fit.rs#L3145-L3149)
for label in crate::benchmarks::cached_preset_labels() {
    let Some(specs) = specs_for_preset_label(label) else { continue };
    let Some(resp) = crate::benchmarks::cached_leaderboard_for_preset(label) else { continue };
    // Compute est/measured ratio using community data...
}

```

This loop calls `cached_preset_labels()` and `cached_leaderboard_for_preset()` from the benchmarks module, applying **MeasuredCommunity** confidence only after confirming no local data exists. The iteration over `cached_preset_labels` at line 3145 and loading via `cached_leaderboard_for_preset` at line 3149 occurs solely as a secondary estimation path.

## Key Source Files and Functions

The precedence logic spans multiple files in the `llmfit-core` crate:

- **[`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs)**: Contains `ModelFit::analyze` and `estimate_basis` implementing the priority logic (lines 3049-3068 and 3145-3149).
- **[`llmfit-core/src/benchmarks.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/benchmarks.rs)**: Provides `cached_preset_labels()` and `cached_leaderboard_for_preset()` for community data retrieval.
- **[`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs)**: Handles canonical slug lookup to match benchmark rows with model entries.

## Summary

- **Local benchmarks take absolute precedence**: When `measured_tps` contains data from a user-run `llmfit bench` command, the system assigns `EstimateConfidence::Measured` and ignores community data.
- **Community presets serve as fallback**: The calibration loop activating at `fit.rs#L3145` only executes when local measurements are absent, assigning `EstimateConfidence::MeasuredCommunity`.
- **Formula estimates rank lowest**: Pure theoretical calculations via `EstimateConfidence::Formula` apply only when neither local nor community data exists.
- **Hierarchy is enforced in `ModelFit::analyze`**: The core logic resides in [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs) with checks at lines 3049-3068 determining which path executes.

## Frequently Asked Questions

### What happens when both local and community benchmarks exist for the same model?

The **local measured benchmarks always win**. As implemented in `ModelFit::analyze` at lines 3049-3068 of [`fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/fit.rs), the presence of `measured_tps` immediately sets `EstimateConfidence::Measured`, causing the fitting routine to skip the community calibration loop entirely regardless of what community data exists on disk.

### How does the system determine which confidence level to assign during estimation?

The assignment follows a strict cascade defined in the `EstimateConfidence` enum. First, the code checks for local measurements at `fit.rs#L3049`. If absent, it falls back to community calibration at line 3145, assigning `MeasuredCommunity`. Only if both sources fail does it default to `Formula` confidence.

### Where does llmfit store the community benchmark data used for fallback?

Community benchmarks reside in the cache accessed through [`llmfit-core/src/benchmarks.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/benchmarks.rs). The `cached_preset_labels()` function retrieves available presets, while `cached_leaderboard_for_preset()` loads the specific leaderboard data for each label, as referenced in the calibration loop at `fit.rs#L3149`.

### Can users force the system to use community presets instead of local benchmarks?

According to the source code in [`fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/fit.rs), there is no override mechanism exposed in `ModelFit::analyze`. The precedence logic hardcodes the check for `self.measured_tps` before considering community data. To use community presets, users must ensure no local benchmark file exists for the target model.