How to Add New Models to the llmfit Catalog: A Step-by-Step Guide

The llmfit model catalog is defined in llmfit-core/data/hf_models.json and extended by editing the scraper script scripts/scrape_hf_models.py, adding your model's Hugging Face repo ID to the TARGET_MODELS list, and running the scraper to regenerate the JSON.

llmfit is an open-source Rust-based utility that simplifies running large language models locally, and its supported model catalog lives in an embedded JSON file. Adding a new model to the llmfit catalog requires modifying a single Python scraper script and rebuilding the project, as documented in the repository's AGENTS.md file. This guide walks through the exact workflow, referencing the actual source files you'll need to touch.

Understanding the llmfit Model Catalog Architecture

The model catalog is stored in llmfit-core/data/hf_models.json, which is embedded into the compiled binary at build time. This JSON file contains metadata for every supported Hugging Face model, including parameter counts, hardware requirements (RAM and VRAM), and repository identifiers.

Rather than maintaining this file by hand, the project relies on a dependency-free Python scraper at scripts/scrape_hf_models.py. According to the repository's AGENTS.md, the scraper is deliberately written using only the Python standard library to avoid polluting the workspace with third-party dependencies.

The scraper maintains two key data structures:

• TARGET_MODELS — a list of Hugging Face repository IDs (e.g., "bigscience/bloom-560m") that define which models get fetched and cataloged.
• FALLBACK — a dictionary with minimal placeholder metadata for gated models that require authentication tokens, allowing the scraper to still generate a valid record.

Step 1: Add Your Model to the Target Models List

Open scripts/scrape_hf_models.py in any editor and locate the TARGET_MODELS list inside the file. Append your model's Hugging Face repository ID to this list. For example, if you want to add the model mistralai/Mistral-7B-v0.1, the entry would look like this inside the script:

TARGET_MODELS = [
    "meta-llama/Llama-3.2-1B",
    "mistralai/Mistral-7B-v0.1",
    # ... existing entries

]

The scraper will then contact the Hugging Face API for this model ID, parse the model card, and compute the hardware requirements using the formulas documented in the repository. The RAM calculation, for instance, uses the formula params * 0.5 bytes / 1024³ * 1.2, which accounts for half-precision memory usage plus a safety margin.

Step 2: (Optional) Add a Fallback for Gated Models

Some models on Hugging Face are gated, meaning they require you to accept license terms before the API will return full metadata. If the model you're adding is gated, the scraper won't get a complete API response, so you need to add a fallback entry to the FALLBACK dictionary in the same script.

The fallback dictionary provides minimal model metadata that the scraper can merge into the generated record when the real API metadata is unavailable. This ensures the catalog still contains a valid, usable placeholder for the model even without direct API access.

Step 3: Run the Scraper to Rebuild the Catalog

Once your model ID is in TARGET_MODELS, execute the scraper from the repository root:

python3 scripts/scrape_hf_models.py

This command does the following:

1. Hits the Hugging Face API for each model in TARGET_MODELS.
2. Parses the model card metadata (parameter count, model type, etc.).
3. Computes RAM/VRAM requirements using the documented formulas.
4. Writes the complete catalog to llmfit-core/data/hf_models.json.

The script overwrites the existing JSON file each time, so you don't need to merge or edit the JSON manually. That said, you can verify the result after scraping:

grep -A2 "Mistral-7B" llmfit-core/data/hf_models.json

This should display the generated entry including the model's computed memory requirements and other metadata.

Step 4: Verify the Generated JSON Entry

After running the scraper, inspect the updated llmfit-core/data/hf_models.json to confirm the new model was added correctly. Look for:

• The correct model repository ID.
• Sensible RAM/VRAM values.
• A valid model type that maps to the runtime's supported architecture.

If the values look off (e.g., RAM is negative or zero), double-check the model card format or consult the fallback mapping for gated models.

Step 5: Rebuild the llmfit Binary

The hf_models.json file is embedded into the final binary via the Cargo build system, as configured in the workspace Cargo.toml. After regenerating the JSON, rebuild the project so the new catalog takes effect:

cargo build

Once the build completes, you can query the new model directly through the CLI:

cargo run -- fit --model "mistralai/Mistral-7B-v0.1"

Key Files to Reference

Here is a summary of every file you'll touch when adding a model — each linked to its location in the repository:

File	Role
scripts/scrape_hf_models.py	Python scraper that populates hf_models.json from Hugging Face API data. Its TARGET_MODELS and FALLBACK structures drive the catalog.
llmfit-core/data/hf_models.json	Embedded JSON catalog of all supported models. Generated by the scraper, not hand-edited.
AGENTS.md	Contribution documentation, including the "Adding a new model" workflow steps.
Cargo.toml (workspace)	Ensures the JSON file is packaged into the final binary at compile time.

Best Practices for Adding Models

• Keep the scraper dependency-free: The project intentionally avoids third-party Python packages. Don't introduce imports like requests — use urllib instead.
• Test with a real model ID: Before adding a model, confirm the Hugging Face repo ID exists and the model card contains the expected metadata (parameter count, architecture).
• Check the RAM/VRAM formulas: The values baked into the JSON drive llmfit's auto-quantization and memory allocation decisions. Verify the computed numbers make sense for the model architecture.
• Re-run the scraper, don't hand-edit JSON: The generated file is the source of truth for builds; manual edits will be lost the next time the scraper runs.

Frequently Asked Questions

Where exactly do I add a new model in the llmfit catalog?

Open scripts/scrape_hf_models.py, find the TARGET_MODELS list, and append the Hugging Face model ID as a string. The list lives at the top of the scraper script. After adding it, run python3 scripts/scrape_hf_models.py and rebuild with cargo build.

Can I edit the hf_models.json file directly instead of using the scraper?

Technically you can add an entry to the JSON manually, but it's strongly discouraged. The scraper is the canonical generator because it computes hardware requirements with the correct formulas. Yellow edits are overwritten the when the scraper runs next.

What should I do if the Hugging Face API return doesn't return complete data for my model?

If your model is gated, add a minimal fallback entry to the FALLBACK dictionary in the scraper. This creates a placeholder record that the runtime can still use. Without a fallback, the scraper will skip the model entirely when API metadata is restricted.

Do I need to install any Python dependencies to run the scraper?

No. According to the repository's AGENTS.md, the script uses only the Python standard library (urllib, json, math, etc.). You only need a working Python 3 environment and outbound network access to the Hugging Face API.

How long does the scraper take to update the catalog for the new model?

Execution time depends on network latency and the number of existing TARGET_MODELS entries. Individual model fetches usually take 1–3 seconds each, so adding one model to an existing list typically completes in under a minute on a reasonable connection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}