How llmfit Handles Custom User Models: A Complete Guide

llmfit enables custom user models by loading a user-specific JSON overlay file that overrides and extends the built-in model catalog without requiring source code modifications.

The llmfit CLI tool from AlexsJones/llmfit maintains a comprehensive catalog of large language models for hardware compatibility checks. Users can augment this catalog with proprietary or specialized models by creating a custom overlay file that the core library merges with its embedded database at runtime according to the logic in llmfit-core/src/models.rs.

Custom Model Configuration Overlay

The system looks for user-defined models in a platform-specific overlay file. By default, llmfit searches for ~/.local/share/llmfit/custom_models.json on Linux systems. This path resolution occurs in the custom_models_file() function defined at lines 1767-1771 of /llmfit-core/src/models.rs.

When the library initializes the model registry, it first loads the embedded catalogue (llmfit-core/data/hf_models.json) compiled into the binary. It then checks for the existence of this overlay path. If found, the library proceeds to parse its contents.

Loading and Parsing User Models

The function load_custom_models_from() handles deserialization of the custom overlay. Located in /llmfit-core/src/models.rs at lines 1778-1790, this function parses the JSON file into a Vec<LlmModel> structure. The schema mirrors exactly that of the embedded catalog, ensuring downstream components process custom entries identically to built-in ones.

Error handling follows a permissive strategy implemented at lines 1815-1823. If the overlay file exists but contains malformed JSON or schema violations, llmfit emits a warning to stderr and continues execution using only the embedded models. This prevents user configuration errors from breaking the toolchain.

Merging Logic and Conflict Resolution

The core merging algorithm, implemented at lines 1814-1820 of llmfit-core/src/models.rs, resolves conflicts through slug-based deduplication. The library generates canonical slugs for both embedded and custom model names, then removes any built-in entries that share identifiers with user-supplied models.

The merge process follows this sequence:

1. Load all embedded models into a mutable vector
2. Parse custom models from the overlay file
3. Build a HashSet<String> of canonical slugs from the custom entries
4. Retain only embedded models whose slugs do not appear in the custom set
5. Extend the vector with the custom LlmModel entries

This approach guarantees that custom definitions always override built-in catalog entries when naming conflicts occur.

// How llmfit merges custom models (simplified)
let mut models = load_embedded_models();                 // built‑in catalog
if let Some(path) = custom_models_file() {               // locate overlay
    if let Ok(custom) = load_custom_models_from(&path) {
        // Remove any embedded models that have the same slug as a custom entry
        let custom_keys: HashSet<String> =
            custom.iter().map(|m| canonical_slug(&m.name)).collect();
        models.retain(|m| !custom_keys.contains(&canonical_slug(&m.name)));
        // Add the custom models
        models.extend(custom);
    } else {
        eprintln!("Warning: skipping custom models: {e}");
    }
}

JSON Schema for Custom Entries

The custom_models.json file must contain a JSON array of model objects. Each object supports the same metadata fields as the embedded catalog, including hardware requirements and quantization specifications.

// Example of a custom model entry (custom_models.json)
[
  {
    "name": "my‑own‑7b‑model",
    "provider": "local",
    "parameter_count": 7_000_000_000,
    "min_ram_gb": 8.0,
    "recommended_ram_gb": 16.0,
    "min_vram_gb": 7.0,
    "quantization": "q4_k_m",
    "context_length": 8192,
    "use_case": "general"
  }
]

Valid fields include parameter count, memory requirements (min_ram_gb, recommended_ram_gb, min_vram_gb), quantization formats, and provider identifiers. The docs/custom-models.md file in the repository provides the complete schema documentation.

Summary

• llmfit supports custom user models through a JSON overlay file at ~/.local/share/llmfit/custom_models.json by default.
• The custom_models_file() function resolves the platform-specific path, while load_custom_models_from() parses entries into Vec<LlmModel>.
• Custom entries override built-in models when their canonical slugs match, with the merge logic at lines 1814-1820 preventing duplicate entries.
• Malformed custom configuration files trigger warnings but do not halt program execution.
• The JSON schema matches the embedded catalog exactly, allowing seamless integration with fit calculation and benchmarking features.

Frequently Asked Questions

Where does llmfit look for custom models on Linux?

By default, llmfit searches for the overlay file at ~/.local/share/llmfit/custom_models.json. The custom_models_file() function in llmfit-core/src/models.rs (lines 1767-1771) resolves this platform-specific path during initialization.

What happens if a custom model has the same name as a built-in model?

The merging algorithm removes conflicting built-in models before appending custom entries. When canonical slugs match between the embedded catalog and user overlay, the custom version takes precedence entirely.

What format should the custom_models.json file use?

The file must contain a JSON array of objects following the same schema as the embedded catalog in llmfit-core/data/hf_models.json. Required fields include name, provider, parameter_count, and memory specifications, while optional fields cover quantization and context length.

Will llmfit crash if my custom models file is invalid?

No. The library implements error isolation in the loading routine at lines 1815-1823. If load_custom_models_from() encounters parsing errors or schema mismatches, it prints a warning to stderr and continues execution using only the embedded catalog.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}