# Bytes per Weight Values for Quantization Types in WhichLLM: Complete Reference Guide

> Discover bytes per weight for WhichLLM quantization types. This guide details the QUANT_BYTES_PER_WEIGHT constant for accurate model storage and VRAM calculations.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: api-reference
- Published: 2026-06-10

---

**WhichLLM defines a comprehensive mapping of quantization formats to their exact bytes-per-weight ratios in the `QUANT_BYTES_PER_WEIGHT` constant, enabling precise calculation of model storage requirements and VRAM usage.**

WhichLLM is an open-source recommendation engine that matches language models to hardware constraints. At the core of its size estimation logic lies a precise lookup table that converts quantization format names into their actual storage costs per parameter, as implemented in `Andyyyy64/whichllm`.

## Where WhichLLM Defines Bytes per Weight Values

The authoritative source for these metrics resides in [`src/whichllm/data/quantization.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/quantization.py). Specifically, lines 4-44 define the constant `QUANT_BYTES_PER_WEIGHT` as a dictionary mapping format identifiers to floating-point storage values.

These figures derive from underlying bit-widths (e.g., 4 bits equals 0.5 bytes) and include overhead for per-block scaling factors where applicable, such as in `MXFP4` and `NVFP4` formats.

## Complete Bytes per Weight Reference Table

The following table lists every quantization type supported by WhichLLM and its corresponding bytes-per-weight value:

| Quantization | Bytes/weight |
|--------------|--------------|
| F32 | 4.0 |
| F16 | 2.0 |
| BF16 | 2.0 |
| Q8_0 | 1.0625 |
| Q6_K | 0.8125 |
| Q5_K_M | 0.6875 |
| Q5_K_S | 0.6875 |
| Q5_0 | 0.625 |
| Q4_K_M | 0.5625 |
| Q4_K_S | 0.5625 |
| Q4_0 | 0.5 |
| Q3_K_M | 0.4375 |
| Q3_K_S | 0.4375 |
| Q3_K_L | 0.4375 |
| Q2_K | 0.3125 |
| IQ4_XS | 0.5 |
| IQ3_XXS | 0.375 |
| IQ2_XXS | 0.25 |
| MXFP4 | 0.53125 |
| NVFP4 | 0.5625 |
| Q1_0 | 0.28 |
| Q2_0 | 0.28 |
| TQ1_0 | 0.21 |
| TQ2_0 | 0.28 |
| IQ1_S | 0.21 |
| IQ1_M | 0.22 |
| IQ2_S | 0.275 |
| IQ2_M | 0.30 |
| IQ3_S | 0.40 |
| IQ3_M | 0.42 |
| IQ3_XS | 0.41 |
| IQ4_NL | 0.5 |

## How WhichLLM Uses Bytes per Weight Data

The quantization constants power the engine's size estimation and ranking systems across three critical modules:

- **Weight Size Estimation**: The `estimate_weight_bytes` function in [`src/whichllm/engine/quantization.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/quantization.py) multiplies model parameter counts by these bytes-per-weight values to predict total storage requirements.
- **VRAM Calculation**: [`src/whichllm/engine/vram.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/vram.py) integrates these metrics to forecast GPU memory requirements, accounting for the actual compressed size of model weights.
- **Model Ranking**: [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) leverages these storage costs to balance the trade-off between quantization compression and output quality when recommending models.

## Practical Code Examples

### Looking Up Specific Quantization Values

```python
from whichllm.data.quantization import QUANT_BYTES_PER_WEIGHT

# Retrieve bytes per weight for Q4_K_M

bytes_per_weight = QUANT_BYTES_PER_WEIGHT["Q4_K_M"]
print(f"Q4_K_M uses {bytes_per_weight:.4f} bytes per weight")

# Output: Q4_K_M uses 0.5625 bytes per weight

```

### Estimating Total Model Size

```python
from whichllm.models.types import ModelInfo
from whichllm.engine.quantization import estimate_weight_bytes

# Create a model with 10 billion parameters

model = ModelInfo(id="example-llm", parameter_count=10_000_000_000)

# Calculate total weight size in bytes

estimated_bytes = estimate_weight_bytes(model, variant=None)
size_gb = estimated_bytes / (1024**3)

print(f"Estimated weight size: {size_gb:.2f} GB")

```

### Custom VRAM Calculator

```python
def required_vram_gb(model, variant):
    """
    Calculate required VRAM including 10% overhead for activations.
    """
    weight_bytes = estimate_weight_bytes(model, variant)
    # Add 10% headroom for inference overhead

    total_bytes = weight_bytes * 1.10
    return total_bytes / (1024**3)  # Convert to GB

```

## Summary

- **WhichLLM defines bytes-per-weight values** in [`src/whichllm/data/quantization.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/quantization.py) within the `QUANT_BYTES_PER_WEIGHT` constant, covering formats from F32 (4.0 bytes) to TQ1_0 (0.21 bytes).
- **The engine uses these metrics** in `estimate_weight_bytes` to calculate model storage requirements and predict VRAM usage.
- **Fractional values** account for compression overhead and per-block scaling factors in advanced formats like MXFP4 and NVFP4.
- **Integration across modules** ensures consistent size estimation from the data layer through the ranking engine.

## Frequently Asked Questions

### What is the bytes per weight value for Q4_K_M quantization in WhichLLM?

According to the source code in [`src/whichllm/data/quantization.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/quantization.py), the `Q4_K_M` quantization format uses **0.5625 bytes per weight**. This represents 4.5 bits per weight on average, accounting for the mixed-bit compression strategy used by this format.

### How does WhichLLM calculate total model size from bytes per weight?

The `estimate_weight_bytes` function in [`src/whichllm/engine/quantization.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/quantization.py) multiplies the model's parameter count by the corresponding bytes-per-weight value from `QUANT_BYTES_PER_WEIGHT`. This yields the total compressed storage size required for the model weights, excluding overhead for activations or KV cache.

### Why do some quantization formats use fractional bytes per weight?

Fractional values occur because quantization compresses weights below the byte boundary. For example, 4-bit quantization yields 0.5 bytes per weight, while formats like `MXFP4` (0.53125 bytes) and `NVFP4` (0.5625 bytes) include additional overhead for per-block scaling factors that slightly increase the average storage cost beyond the raw bit-width conversion.

### Where can I find the complete list of quantization types supported by WhichLLM?

The complete dictionary mapping quantization names to bytes-per-weight values is defined in [`src/whichllm/data/quantization.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/quantization.py) (lines 4-44). This file contains the `QUANT_BYTES_PER_WEIGHT` constant that serves as the single source of truth for all quantization metrics used throughout the WhichLLM codebase.