How llmfit-core Loads Schema-v1 Hardware Profiles and Applies Capacity Overrides to SystemSpecs

llmfit-core embeds bundled JSON profiles at build time, loads user-supplied overrides from ~/.local/share/llmfit/hardware, merges them into a shadowed catalog, and applies capacity overrides to SystemSpecs via with_profile_capacity while injecting synthetic GPU pools for unified-memory configurations.

The hardware profiling system in llmfit-core enables precise performance modeling by allowing developers to override auto-detected system capacities with curated schema-v1 hardware profiles. According to the AlexsJones/llmfit source code, the implementation spans build-time embedding, runtime discovery, and selective capacity injection into the SystemSpecs struct.

Understanding the Hardware Profile Architecture

Schema-v1 Definition and Validation

Hardware profiles in llmfit-core conform to schema-v1, defined in llmfit-core/data/hardware/schema.json. Each profile specifies total_ram_gb, unified_memory flags, and estimator inputs including bandwidth and TFLOPS. The HardwareProfile::validate() method enforces these constraints during parsing, while HardwareProfile::parse() uses serde_json::from_str with lenient deserialization to tolerate unknown keys for forward compatibility.

Embedded vs. User-Supplied Profiles

The system maintains two distinct profile sources. Bundled profiles ship with the binary and are embedded via include_str!() as EMBEDDED_PROFILES_JSON in llmfit-core/src/hwprofile.rs. User profiles reside in ~/.local/share/llmfit/hardware by default, or the path specified by the LLMFIT_HARDWARE_PROFILES environment variable. User-provided profiles shadow bundled entries when they share the same name field, enabling rapid iteration without recompiling.

Loading Pipeline for Schema-v1 Hardware Profiles

Build-Time Embedding of Bundled Profiles

During compilation, the workspace build.rs collects every data/hardware/*.json file and writes a consolidated hardware_profiles.json into OUT_DIR. The embedded() function in llmfit-core/src/hwprofile.rs lazily parses this JSON through a OnceLock, validating each entry with HardwareProfile::validate() and caching the resulting Vec<HardwareProfile>. Invalid entries are silently dropped to prevent CI failures.

// From llmfit-core/src/hwprofile.rs
// EMBEDDED_PROFILES_JSON is generated by build.rs
pub fn embedded() -> &'static [HardwareProfile] {
    EMBEDDED_PROFILES.get_or_init(|| {
        // Parse and validate bundled profiles
    })
}

Runtime Loading of User Profiles

The user_profile_dir() helper resolves the hardware profile directory, checking the LLMFIT_HARDWARE_PROFILES environment variable before falling back to the XDG data directory. The load_file() function reads individual JSON files, validates them, and ensures the profile's internal name matches the filename stem via check_name_matches_stem. Errors accumulate in ProfileCatalog.errors without halting execution.

// Resolves ~/.local/share/llmfit/hardware or env override
let user_dir = user_profile_dir()?;
// Each file validated and checked against its stem
let profile = load_file(path)?;

Catalog Construction and Shadowing Behavior

The catalog_in() function constructs the final profile registry by first inserting all embedded profiles, then walking the user directory. When a user profile has the same name as a bundled entry, the implementation removes the bundled version before inserting the user version. The final list is sorted alphabetically by name.

// In llmfit-core/src/hwprofile.rs lines 48-84
// User profiles shadow bundled ones by name
if let Some(pos) = list.iter().position(|p| p.name == profile.name) {
    list.remove(pos);
}
list.push(profile);
list.sort_by(|a, b| a.name.cmp(&b.name));

Resolving and Applying Capacity Overrides

Profile Resolution via resolve() and find()

The public resolve() API in llmfit-core/src/hwprofile.rs accepts either a profile name or a filesystem path. The looks_like_path() helper distinguishes between identifiers and paths; paths are loaded directly via load_file(), while names trigger a catalog lookup through catalog().find(). This dual-mode resolution supports both bundled references and ad-hoc configuration files.

SystemSpecs Capacity Injection

Once resolved, HardwareProfile::apply_to_specs() forwards the profile's total_ram_gb and unified_memory flag to SystemSpecs::with_profile_capacity(). This method creates a new SystemSpecs instance where the detected RAM is replaced, available_ram_gb is recomputed, and for unified-memory architectures, the system sets has_gpu = true while mirroring the RAM capacity into gpu_vram_gb and total_gpu_vram_gb. This synthetic pool allows unified-memory chips like Apple Silicon or AMD APU configurations to be modeled as GPU-capable without discrete VRAM.

// From llmfit-core/src/hardware.rs
pub fn with_profile_capacity(total_ram_gb: u64, unified_memory: bool) -> Self {
    // Replaces detected RAM, recomputes available memory
    // Sets has_gpu=true and mirrors RAM to VRAM fields if unified_memory
}

Estimator Configuration Overrides

Beyond capacity, profiles override calculation parameters through apply_to_config(), which mutates a CalcConfig instance with profile-specific bandwidth, TFLOPS, efficiency ratings, and run-mode factors. The convenience method apply() executes both capacity injection and configuration override in a single call.

let mut cfg = CalcConfig::default();
// Apply both capacity and estimator overrides
let new_specs = profile.apply(specs, &mut cfg);

Practical Implementation Example

The following example demonstrates the complete workflow: detecting hardware, resolving a schema-v1 profile by name, and applying both capacity overrides and estimator configurations.

use llmfit_core::{
    hardware::SystemSpecs,
    fit::CalcConfig,
    hwprofile::{resolve, LoadedProfile},
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Detect current hardware configuration
    let specs = SystemSpecs::detect();
    
    // Resolve bundled profile "ryzen-ai-max-plus-395"
    let LoadedProfile { profile, .. } = resolve("ryzen-ai-max-plus-395")?;
    
    // Apply capacity overrides (handles unified-memory synthesis)
    let specs = profile.apply_to_specs(specs);
    
    // Apply estimator inputs (bandwidth, TFLOPS, etc.)
    let mut cfg = CalcConfig::default();
    profile.apply_to_config(&mut cfg);
    
    // specs and cfg now reflect the profile's declared capabilities
    println!("Adjusted RAM: {} GB", specs.total_ram_gb);
    println!("GPU available: {}", specs.has_gpu);
    
    Ok(())
}

For one-shot application of both steps:

let mut cfg = CalcConfig::default();
let specs = profile.apply(specs, &mut cfg);

Summary

• Schema-v1 profiles are defined in data/hardware/schema.json and parsed with forward-compatible leniency in llmfit-core/src/hwprofile.rs.
• Build-time embedding via build.rs and include_str!() makes bundled profiles available through the embedded() function.
• User override directory defaults to ~/.local/share/llmfit/hardware or uses the LLMFIT_HARDWARE_PROFILES environment variable.
• Shadowing behavior ensures user profiles replace bundled entries with matching names during catalog construction in catalog_in().
• Capacity injection occurs through SystemSpecs::with_profile_capacity(), which replaces detected RAM and synthesizes GPU VRAM fields for unified-memory configurations.
• Dual override system separates capacity adjustments (apply_to_specs) from estimator tuning (apply_to_config), combinable via apply().

Frequently Asked Questions

How does llmfit-core handle invalid user-provided hardware profiles?

Invalid profiles discovered in the user directory are collected into ProfileCatalog.errors during the loading phase but do not halt execution or prevent valid profiles from loading. Strict validation using validate_strict is reserved for the CLI validation command, while runtime parsing remains lenient to maintain forward compatibility.

What happens when a user profile has the same name as a bundled profile?

The catalog implements a shadowing rule where user-provided profiles override bundled ones. During catalog_in(), if a user profile's name field matches an existing entry, the bundled version is removed before inserting the user version, allowing customized definitions without binary recompilation.

How does unified memory detection affect GPU capacity reporting?

When a schema-v1 profile specifies unified_memory: true, SystemSpecs::with_profile_capacity() sets has_gpu = true and mirrors the total_ram_gb value into both gpu_vram_gb and total_gpu_vram_gb. This creates a synthetic GPU memory pool representing shared system memory, essential for accurate fitting calculations on integrated graphics architectures.

Can I load a hardware profile directly from a file path instead of the catalog?

Yes. The resolve() function in llmfit-core/src/hwprofile.rs uses looks_like_path() to detect filesystem paths. When provided a path-like string, it bypasses the catalog and loads the profile directly via load_file(), enabling ad-hoc testing of configuration files without placing them in the standard user directory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}