# Supported Bit Widths for TurboQuant in MLX-VLM: Integer and Half-Integer Values Explained

> Explore TurboQuant's supported bit widths in MLX-VLM discover integer and half-integer options. Learn about the minimum accepted value and how fractional values are handled to optimize your models.

- Repository: [Prince Canuma/mlx-vlm](https://github.com/Blaizzy/mlx-vlm)
- Tags: deep-dive
- Published: 2026-04-05

---

**TurboQuant supports bit widths that are integers (1, 2, 3...) or half-integers (1.5, 2.5, 3.5...) with a minimum value of 1, rejecting all other fractional values with a `ValueError`.**

TurboQuant is a quantization method implemented in the **mlx-vlm** repository for efficient vision-language model inference. When configuring key-value cache quantization, developers must specify valid `kv_bits` parameters that align with the codec's precision requirements. The implementation explicitly validates these values at instantiation to ensure compatibility with the underlying quantization kernels.

## Allowed Bit Width Values for TurboQuant

TurboQuant accepts two distinct categories of bit widths, both requiring a minimum value of **1**:

- **Integer values**: Whole numbers such as `1`, `2`, `3`, `4`, and higher
- **Half-integer values**: Numbers ending in `.5` such as `3.5`, `4.5`, `5.5`, providing finer granularity between whole-bit precisions

Any value below 1 or containing fractional components other than `.5` will trigger validation errors. For example, `3.0`, `4`, and `5.5` are valid configurations, while `0.5` or `3.25` are rejected.

## Implementation Details in MLX-VLM Source Code

The validation logic resides in the [`mlx_vlm/turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/turboquant.py) module, where the constructor performs strict type checking before allowing instantiation.

### Constructor Validation in turboquant.py

At line 3453 of [`mlx_vlm/turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/turboquant.py), the implementation validates the `kv_bits` parameter and raises a descriptive error for unsupported values:

```python
raise ValueError("TurboQuant currently supports integer and .5 bit-widths, got {bits}.")

```

This check ensures that the codec receives only precision values it can encode efficiently. The validation occurs immediately upon creating a `TurboQuant` instance, preventing runtime errors during the quantization process.

### Unit Test Coverage

The test suite in [`mlx_vlm/tests/test_turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/tests/test_turboquant.py) confirms both integer and half-integer support through explicit test cases at lines 79 and 98:

```python

# Half-integer bit width

kv_bits=3.5

# Integer bit width  

kv_bits=3.0

```

These tests verify that the quantization pipeline accepts both `3.5` (half-integer) and `3.0` (integer) without raising exceptions, validating the core functionality across different precision levels.

## Practical Usage Examples

When initializing TurboQuant for KV cache quantization, use only supported values:

```python
from mlx_vlm.turboquant import TurboQuant

# Valid configurations

quant_4bit = TurboQuant(kv_bits=4)      # Integer

quant_3_5bit = TurboQuant(kv_bits=3.5)  # Half-integer

quant_2bit = TurboQuant(kv_bits=2.0)    # Integer as float

# Invalid - will raise ValueError

try:
    invalid_quant = TurboQuant(kv_bits=3.25)
except ValueError as e:
    print(e)  # TurboQuant currently supports integer and .5 bit-widths, got 3.25.

```

## Summary

- TurboQuant requires `kv_bits` values to be **integers** or **half-integers** (integer + 0.5)
- The **minimum supported value is 1**; anything lower raises a `ValueError`
- Valid examples include `1`, `2.5`, `3`, `4.5`, `8`, etc.
- Invalid values like `3.25` or `0.5` trigger explicit validation errors in [`mlx_vlm/turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/turboquant.py)
- Unit tests in [`test_turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/test_turboquant.py) verify both integer and half-integer code paths

## Frequently Asked Questions

### Does TurboQuant support 4-bit quantization?

Yes, `kv_bits=4` is fully supported as a valid integer bit width. This is one of the most common configurations for balancing memory efficiency and model accuracy in vision-language models.

### What happens if I try to use 3.25 bits?

The constructor raises a `ValueError` with the message: *"TurboQuant currently supports integer and .5 bit-widths, got 3.25."* This strict validation prevents the codec from attempting to quantize to unsupported precision levels that lack kernel implementations.

### Is there a performance difference between integer and half-integer bit widths?

While both types are supported, half-integer values like `3.5` provide finer granularity for memory-accuracy tradeoffs. The performance characteristics depend on the specific hardware acceleration available in MLX, but both formats undergo the same validation pipeline in [`mlx_vlm/turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/turboquant.py).

### What is the minimum supported bit width for TurboQuant?

The minimum value is **1 bit**. Attempting to instantiate TurboQuant with `kv_bits=0.5` or any value less than 1 will raise a `ValueError: TurboQuant requires kv_bits >= 1`, as enforced by the validation logic in the source code.