# How to Add Custom Models to llmfit Without Recompiling

> Learn how to add custom models to llmfit without recompiling. Use a custom_models.json overlay file to easily integrate new models with the runtime loader.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: how-to-guide
- Published: 2026-07-23

---

**You can add custom models to llmfit without recompiling by creating a [`custom_models.json`](https://github.com/AlexsJones/llmfit/blob/main/custom_models.json) overlay file that the runtime loader merges with the embedded catalog, optionally overriding the path via the `LLMFIT_CUSTOM_MODELS` environment variable.**

llmfit is a Rust-based CLI tool that matches Large Language Models (LLMs) to your hardware constraints. While the binary ships with embedded JSON catalogs for Hugging Face and ONNX models, the source code in [`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs) implements a runtime overlay mechanism that lets you extend or override these definitions without rebuilding the binary.

## Understanding the Runtime Overlay Mechanism

The `ModelDatabase::new()` constructor in [`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs) (lines 1551-1574) implements a two-phase loading strategy. First, it deserializes the embedded [`hf_models.json`](https://github.com/AlexsJones/llmfit/blob/main/hf_models.json) and [`onnx_models.json`](https://github.com/AlexsJones/llmfit/blob/main/onnx_models.json) catalogs compiled into the binary. Then it calls `custom_models_file()` to determine the overlay path and `load_custom_models_from()` to parse any user-defined entries.

If a custom entry shares the same **canonical slug** as an embedded model, the custom definition **replaces** the built-in one. Otherwise, the entry is appended to the list. This merge happens entirely at runtime, meaning the binary reads your modifications from disk every time it executes.

## Locating the Custom Models File

By default, llmfit looks for the overlay file using the `custom_models_file` helper function (lines 1414-1422). On Linux, this resolves to `~/.local/share/llmfit/custom_models.json` within the update cache directory. The path varies by platform following OS conventions for application data.

You can override this location by setting the **`LLMFIT_CUSTOM_MODELS`** environment variable to an absolute path before invoking the CLI:

```bash
export LLMFIT_CUSTOM_MODELS="/path/to/my_custom_models.json"
llmfit fit --perfect

```

The `load_custom_models_from` function (lines 1428-1439) handles the actual disk I/O. If the file does not exist, the loader silently continues with the embedded catalog only. If the file contains malformed JSON, the program emits a warning to stderr but does not crash, ensuring robustness against editing errors.

## Creating a Valid custom_models.json

Your overlay must follow the same schema as the embedded [`hf_models.json`](https://github.com/AlexsJones/llmfit/blob/main/hf_models.json). The JSON file should contain an array of model objects. Required fields include `name`, `provider`, `parameter_count`, `min_ram_gb`, `recommended_ram_gb`, `min_vram_gb`, `quantization`, `context_length`, and `use_case`.

Optional metadata such as `capabilities`, `languages`, `license`, and hardware flags like `is_moe` improve filtering in the TUI but can be omitted.

Create the file with a structure like this:

```json
[
  {
    "name": "my-org/MyCustom-7B",
    "provider": "my-org",
    "parameter_count": "7B",
    "min_ram_gb": 12.5,
    "recommended_ram_gb": 24.0,
    "min_vram_gb": 8.5,
    "quantization": "Q4_K_M",
    "context_length": 4096,
    "use_case": "general",
    "is_moe": false,
    "capabilities": ["chat", "completion"],
    "languages": ["en"],
    "format": "gguf",
    "hf_downloads": 0,
    "hf_likes": 0,
    "license": "MIT"
  }
]

```

Save this to the default location or the path specified by `LLMFIT_CUSTOM_MODELS`.

## Step-by-Step: Adding Your First Custom Model

Follow these steps to register a new model without touching the Rust source:

1. **Create the cache directory** if it does not exist:

   ```bash
   mkdir -p ~/.local/share/llmfit
   ```

2. **Write the overlay file** using the schema above. Ensure valid JSON syntax; the parser will reject trailing commas or unquoted keys.

3. **Verify the merge** by listing models matching your entry:

   ```bash
   llmfit list "MyCustom"
   ```

   The output should display your custom entry with the specified VRAM and RAM requirements.

4. **Override existing models** (optional) by using the same `name` slug as a built-in model. Your custom values for `min_vram_gb` or `quantization` will take precedence in the [`llmfit-core/src/fit.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/fit.rs) calculations.

## Summary

- **Runtime overlay**: llmfit merges [`custom_models.json`](https://github.com/AlexsJones/llmfit/blob/main/custom_models.json) with embedded catalogs at startup via `ModelDatabase::new()`.
- **File location**: Default is the platform-specific cache directory; override with `LLMFIT_CUSTOM_MODELS`.
- **Schema compliance**: Match the [`hf_models.json`](https://github.com/AlexsJones/llmfit/blob/main/hf_models.json) structure with required fields like `name`, `provider`, and `min_ram_gb`.
- **Safe loading**: Missing files are ignored; malformed JSON triggers warnings but not crashes.
- **Override capability**: Custom entries with matching slugs replace built-in definitions.

## Frequently Asked Questions

### What happens if my custom_models.json contains syntax errors?

The `load_custom_models_from` function catches parse errors and logs a warning to stderr, then continues execution using only the embedded model catalog. The program will not crash, but your custom models will not appear in the list until the JSON is valid.

### Can I override existing Hugging Face models with custom parameters?

Yes. If your custom entry uses the same `name` (canonical slug) as an embedded model from [`hf_models.json`](https://github.com/AlexsJones/llmfit/blob/main/hf_models.json), your version **replaces** the built-in definition entirely. This is useful for adjusting VRAM estimates or quantization defaults for specific hardware configurations.

### Where is the default custom_models.json path on Windows and macOS?

The `custom_models_file` function resolves platform-specific cache directories. On macOS, this typically resides in `~/Library/Application Support/llmfit/custom_models.json`, while Windows uses the `%LOCALAPPDATA%\llmfit\custom_models.json` path. You can always override these defaults using the `LLMFIT_CUSTOM_MODELS` environment variable.

### Do I need to restart llmfit after editing the overlay file?

No recompilation is necessary, but you must restart the `llmfit` process. The JSON file is read once during `ModelDatabase` initialization at startup. Changes made while the TUI is running will not appear until the next invocation.