# How Does whichllm Generate Synthetic GGUF Variants for LLMs? A Code-Level Breakdown

> Discover how whichllm creates synthetic GGUF variants from Safetensors files by estimating sizes, enabling better LLM model ranking even without real GGUF conversions. Get the code-level details.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: internals
- Published: 2026-06-09

---

**whichllm generates synthetic GGUF variants by estimating file sizes from parameter counts and bytes-per-weight constants when a model repository only provides Safetensors files, allowing the ranking engine to score models that lack real GGUF conversions.**

The open-source tool **whichllm** solves the problem of missing GGUF conversions by simulating them. When a model repository ships only Safetensors weights, the engine synthesizes plausible GGUF variants so they can still be ranked for size, speed, and quality. This article explains exactly how the `Andyyyy64/whichllm` ranking pipeline generates these synthetic estimates directly from the source code.

## Detecting Missing GGUF Files in the Ranker

In [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py), the method `_iter_candidate_variants` inspects `model.gguf_variants` to decide whether to use real or synthetic data. If that list is empty, the engine falls back to synthetic generation at lines 150–156. This fallback ensures that Safetensors-only models are not silently discarded during ranking.

## Eligibility Filters for Synthetic Generation

### Official Organization Check

The helper `_synthesize_variants_for_official_repo` first verifies that the model owner is trusted. It splits `model.id` and checks the namespace against `_OFFICIAL_ORGS`, defined in [`src/whichllm/constants.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/constants.py). If the namespace is not on the allow-list, synthetic generation is skipped entirely.

### Pre-Quantized Repository Exclusion

The same function also excludes repositories that already advertise pre-quantized formats such as `-AWQ` or `-GPTQ`. A regex `_PREQUANTIZED_REPO_RE` filters these out to avoid creating redundant synthetic estimates, as seen in lines 109–125 of [`ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/ranker.py).

## Choosing Quantization Levels and Estimating Sizes

### Hard-Coded Synthetic Quant Types

whichllm uses a fixed tuple `_SYNTHETIC_QUANTS = ("Q3_K_M", "Q4_K_M", "Q5_K_M", "Q6_K", "Q8_0")` to represent the quantization profiles the community typically publishes shortly after release. This tuple is defined near lines 102–103 in [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py).

### Parameter-to-Size Calculation

For each quant type, the approximate file size is computed as `model.parameter_count * QUANT_BYTES_PER_WEIGHT[quant]`. The mapping `QUANT_BYTES_PER_WEIGHT` lives in [`src/whichllm/constants.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/constants.py) and defaults to `0.5625` bytes per weight when a specific quant is not mapped. This calculation occurs at lines 135–141 of the ranker.

## Emitting and Scoring Synthetic Variants

### The GGUFVariant Dataclass

Each synthetic result is wrapped in a `GGUFVariant` object defined in [`src/whichllm/models/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/types.py). The dataclass stores `filename`, `quant_type`, and `file_size_bytes`.

### Integration into the Ranking Pipeline

After generation, synthetic variants flow through the same compatibility and performance scoring as real GGUF files. The engine marks them with a note via logic in [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py)—specifically `_looks_synthetic_gguf`—so users know the values are estimates rather than measured binaries.

## Code Walkthrough: `_synthesize_variants_for_official_repo`

Here is the core logic from [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) that produces synthetic variants:

```python
def _synthesize_variants_for_official_repo(model: ModelInfo,
                                            quant_filter_upper: str | None) -> list[GGUFVariant]:
    # ① Only official orgs & not already pre-quantized

    if model.id.split("/", 1)[0] not in _OFFICIAL_ORGS:
        return []
    if _PREQUANTIZED_REPO_RE.search(model.id):
        return []

    variants = []
    for quant in _SYNTHETIC_QUANTS:
        if quant_filter_upper and quant != quant_filter_upper:
            continue
        # ② Approximate size = params × bytes-per-weight

        bpw = QUANT_BYTES_PER_WEIGHT.get(quant, 0.5625)
        variants.append(
            GGUFVariant(
                filename=f"{model.name}.{quant}.gguf",
                quant_type=quant,
                file_size_bytes=int(model.parameter_count * bpw),
            )
        )
    return variants

```

The caller, `_iter_candidate_variants`, substitutes this list whenever a model has no real GGUF files, ensuring the ranking pipeline never drops an otherwise runnable candidate.

## Practical Usage Examples

### Ranking a Model Without GGUF Files

When you pass a `ModelInfo` with an empty `gguf_variants` list to `rank_models`, whichllm automatically injects synthetic estimates:

```python
from whichllm.engine.ranker import rank_models
from whichllm.models.types import ModelInfo
from whichllm.hardware.types import HardwareInfo

model = ModelInfo(
    id="Qwen/Qwen3.6-27B",
    family_id="qwen3-6b-gguf",
    name="Qwen3.6-27B",
    parameter_count=27_000_000_000,
    gguf_variants=[],
)

hw = HardwareInfo(os="linux", gpus=[...])
ranked = rank_models([model], hw, top_n=5)

# ranked[0].gguf_variant is a synthetic GGUFVariant, e.g.:

# GGUFVariant(filename='Qwen3.6-27B.Q4_K_M.gguf', quant_type='Q4_K_M', ...)

```

### Generating Synthetic Variants Directly

You can also call the internal helper directly for testing or debugging:

```python
from whichllm.engine.ranker import _synthesize_variants_for_official_repo
from whichllm.models.types import ModelInfo

model = ModelInfo(
    id="meta-llama/Llama-2-7B",
    family_id="llama-2-7b",
    name="Llama-2-7B",
    parameter_count=7_000_000_000,
)

synthetic = _synthesize_variants_for_official_repo(model, None)

# Returns list[GGUFVariant] for Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, and Q8_0

```

## Key Files in the Synthetic GGUF Pipeline

- [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) — Hosts `_synthesize_variants_for_official_repo` and the `_iter_candidate_variants` fallback logic.
- [`src/whichllm/models/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/types.py) — Defines the `GGUFVariant` dataclass.
- [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py) — Adds synthetic notes and adjusts speed uncertainty via `_looks_synthetic_gguf`.
- [`src/whichllm/engine/types.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/types.py) — Supplies `CompatibilityResult` for ranking flow integration.
- [`src/whichllm/constants.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/constants.py) — Stores `QUANT_BYTES_PER_WEIGHT`, `_OFFICIAL_ORGS`, and `_SYNTHETIC_QUANTS`.

## Summary

- whichllm creates **synthetic GGUF variants** when a repository only provides Safetensors files.
- Eligibility is restricted to **official organizations** and excludes **pre-quantized** repositories.
- File sizes are estimated using **`parameter_count * QUANT_BYTES_PER_WEIGHT`** with a default of `0.5625` bytes per weight.
- The fixed quant set **`("Q3_K_M", "Q4_K_M", "Q5_K_M", "Q6_K", "Q8_0")`** covers common community conversions.
- Synthetic variants are wrapped in **`GGUFVariant`** objects and scored identically to real GGUF files, but flagged as estimates.

## Frequently Asked Questions

### What is a synthetic GGUF variant in whichllm?

A synthetic GGUF variant is a placeholder object that simulates a quantized GGUF file for a model that does not publish one. It is generated by the ranking engine so that size and compatibility constraints can still be evaluated as if a real conversion existed.

### Which quantization types does whichllm synthesize?

The engine synthesizes five standard types defined in `_SYNTHETIC_QUANTS`: **Q3_K_M**, **Q4_K_M**, **Q5_K_M**, **Q6_K**, and **Q8_0**. These reflect the most common post-release GGUF quantization profiles published by the community.

### How accurate are the synthetic GGUF size estimates?

Accuracy depends on the **`QUANT_BYTES_PER_WEIGHT`** constant for each quant type, with a fallback default of `0.5625` bytes per weight. The estimate is deterministic and fast, but it remains an approximation until an actual GGUF conversion is published and measured.

### Why does whichllm only generate synthetic variants for official repositories?

The helper `_synthesize_variants_for_official_repo` gates synthetic creation on the `_OFFICIAL_ORGS` list to reduce the risk of estimating invalid or misleading variants from unofficial forks. This policy keeps the ranking results focused on trusted model publishers.