# How to Add Custom Models to llmfit's Catalog Without Recompiling

> Easily add custom llmfit models without recompiling. Create a JSON overlay file in your user data directory for automatic integration. Enhance your llmfit experience today.

- Repository: [Alex Jones/llmfit](https://github.com/AlexsJones/llmfit)
- Tags: how-to-guide
- Published: 2026-08-20

---

**Create a JSON overlay file at your platform's user data directory (`~/.local/share/llmfit/custom_models.json` on Linux) and llmfit will automatically merge your custom entries into the built-in catalog at runtime.**

The llmfit tool ships with an embedded catalog of models, but you don't need to rebuild the Rust binary to extend it. The `llmfit-core` library provides a **runtime overlay mechanism** that reads user-defined models from a JSON file and merges them into the catalog. This approach lets you add proprietary, fine-tuned, or newly released models without modifying source code or waiting for upstream updates.

## Where llmfit Looks for Custom Models

The function `custom_models_file()` in [`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs) (lines [1466-1470](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs#L1466-L1470)) returns the platform-specific path:

- **Linux**: `~/.local/share/llmfit/custom_models.json`
- **macOS**: `~/Library/Application Support/llmfit/custom_models.json`
- **Windows**: `%APPDATA%\Roaming\llmfit\custom_models.json`

This path is constructed using the same `cache_dir()` utility found in [`llmfit-core/src/update.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/update.rs), ensuring consistency with other user-level data storage.

## How the Overlay Loading Works

When the model database initializes, `load_custom_models_from()` (lines [1477-1519](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs#L1477-L1519)) handles the merge logic:

- **File missing**: Silently ignored—no warning, no error
- **Parse success**: Custom entries **replace** any built-in model with the same slug (canonicalized name)
- **Parse failure**: Warning emitted, but llmfit continues with the embedded catalog

This replacement behavior means you can override existing entries to correct parameters or update quantization formats without touching the source.

## Required JSON Schema for Custom Models

Each entry must match the `LlmModel` struct defined around line [961-970](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs#L961-L970). The minimal required structure:

```json
{
  "name": "my-custom-model",
  "provider": "local",
  "parameter_count": 7000000000,
  "min_ram_gb": 4.5,
  "recommended_ram_gb": 9.0,
  "min_vram_gb": 3.0,
  "quantization": "Q4_K_M",
  "context_length": 8192,
  "use_case": "general"
}

```

Key field definitions based on the source:

- **`name`**: Unique identifier used as the slug (canonicalized internally)
- **`provider`**: Typically `"local"` for custom GGUF files or alternative backends
- **`parameter_count`**: Integer count of model parameters
- **`min_ram_gb` / `recommended_ram_gb`**: System memory thresholds for analysis
- **`min_vram_gb`**: GPU memory floor for fit calculations
- **`quantization`**: Format string (e.g., `"Q4_K_M"`, `"Q5_K_S"`, `"FP16"`)
- **`context_length`**: Maximum token context window
- **`use_case`**: Category string for filtering (`"general"`, `"coding"`, `"chat"`, etc.)

## Step-by-Step: Adding Your First Custom Model

### 1. Create the overlay directory and file

```bash

# Linux/macOS

mkdir -p ~/.local/share/llmfit
touch ~/.local/share/llmfit/custom_models.json

```

```powershell

# Windows PowerShell

New-Item -ItemType Directory -Force "$env:APPDATA\llmfit"
New-Item -ItemType File "$env:APPDATA\llmfit\custom_models.json"

```

### 2. Populate [`custom_models.json`](https://github.com/AlexsJones/llmfit/blob/main/custom_models.json) with one or more models

```json
[
  {
    "name": "my-custom-7b-coder",
    "provider": "local",
    "parameter_count": 7000000000,
    "min_ram_gb": 4.5,
    "recommended_ram_gb": 9.0,
    "min_vram_gb": 3.0,
    "quantization": "Q4_K_M",
    "context_length": 16384,
    "use_case": "coding"
  },
  {
    "name": "my-tiny-1.5b-model",
    "provider": "local",
    "parameter_count": 1500000000,
    "min_ram_gb": 1.2,
    "recommended_ram_gb": 2.5,
    "min_vram_gb": 0.8,
    "quantization": "Q4_K_M",
    "context_length": 4096,
    "use_case": "general"
  }
]

```

### 3. Verify custom models appear in the catalog

```bash

# List all models and filter for your custom entry

llmfit --list-models | grep my-custom

```

The output should show both `my-custom-7b-coder` and `my-tiny-1.5b-model` alongside llmfit's built-in HuggingFace models.

## Overriding Built-In Models

To replace an existing catalog entry, use the same `name` field as the embedded model. For example, to override the quantization or RAM estimates for a popular model, check its slug in the source [`hf_models.json`](https://github.com/AlexsJones/llmfit/blob/main/hf_models.json) or via `--list-models`, then create a custom entry with matching `name`. The overlay takes precedence during the merge.

## Why No Recompilation Is Required

The custom model overlay is **pure runtime configuration**:

- `llmfit-core` deserializes the JSON at database initialization
- No `include_str!` or compile-time embedding of custom data
- The binary contains only the logic to locate and parse the file

This design separates code from data, enabling rapid iteration on model parameters without Cargo, `rustc`, or CI pipelines.

## Key Source Files Reference

| File | Purpose | Link |
|------|---------|------|
| [`llmfit-core/src/models.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs) | `LlmModel` struct, `custom_models_file()`, `load_custom_models_from()` | [models.rs](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/models.rs) |
| [`llmfit-core/src/update.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/update.rs) | `cache_dir()` helper for platform paths | [update.rs](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/update.rs) |
| [`llmfit-core/src/analysis.rs`](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/analysis.rs) | Consumes the merged model list for fit analysis | [analysis.rs](https://github.com/AlexsJones/llmfit/blob/main/llmfit-core/src/analysis.rs) |

## Summary

- **Location**: Platform-specific user data directory (`~/.local/share/llmfit/` on Linux)
- **Mechanism**: JSON overlay file [`custom_models.json`](https://github.com/AlexsJones/llmfit/blob/main/custom_models.json) merged at runtime by `load_custom_models_from()`
- **Behavior**: Custom entries replace same-named built-in models; missing files ignored silently
- **Schema**: Match the `LlmModel` struct fields—`name`, `provider`, memory specs, `quantization`, `context_length`, `use_case`
- **No rebuild**: The binary reads the file dynamically; edit and restart llmfit to apply changes

## Frequently Asked Questions

### Can I use the overlay to remove or disable built-in models?

No. The overlay only adds or replaces entries—it cannot delete from the embedded catalog. To effectively hide a model, replace its entry with implausible parameters (e.g., `min_ram_gb: 9999`) so it fails fit criteria, or filter at the CLI level.

### Does llmfit validate the custom JSON schema strictly?

Partially. `load_custom_models_from()` uses Serde deserialization; missing required fields or type mismatches trigger a parse warning and skip the file. Extra fields are generally ignored per Rust's typical deserialization behavior, but rely only on documented `LlmModel` fields for forward compatibility.

### Will my custom models persist across llmfit updates?

Yes. The overlay lives in your user data directory, not the installation path. Updating the binary via `cargo install` or package managers leaves `~/.local/share/llmfit/` untouched.

### Can I specify remote (API-based) models instead of local GGUF files?

The `provider` field supports different backends, but as of the current source, `"local"` is the primary tested path for custom entries. API-based providers like OpenAI or Anthropic models are typically built into the embedded catalog. Check [`models.rs`](https://github.com/AlexsJones/llmfit/blob/main/models.rs) for the latest `Provider` enum variants if experimenting with remote custom endpoints.