# What Is CQ2-Bit Quantization and How Is It Used in Needle 2?

> Discover CQ2-Bit Quantization, a learned codebook compression technique that reduces model size by representing weights with just 2 bits. Learn how Needle 2 leverages this to pack millions of parameters into MBs.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-29

---

**CQ2-bit quantization is a learned codebook compression technique that packs 45 million parameters into 14 MB by representing each weight with only 2 bits, utilizing Hadamard rotation to preserve model quality.**

CQ2-bit quantization serves as the flagship compression engine in the Needle 2 framework, enabling deployment of large language models on memory-constrained devices without sacrificing accuracy. As part of the Cactus Quants family, this method reduces weight precision to 2 bits per value through vector quantization and orthogonal transformation. The complete implementation resides in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) and supports both uniform and mixed-precision quantization strategies.

## How CQ2-Bit Quantization Works

The CQ2-bit algorithm compresses neural network weights through a three-stage pipeline that decorrelates dimensions before applying scalar quantization.

### Hadamard Rotation and Decorrelation

First, the algorithm applies a fixed Walsh-Hadamard matrix transform to weight groups. In [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py), the `cq_quantize` function (starting at line 41) generates the rotation matrix via `_cq_hadamard_np(g)` and applies it through matrix multiplication:

```python
H = jnp.asarray(_cq_hadamard_np(g))
rot = groups @ H

```

This rotation decorrelates the weight dimensions, transforming the distribution into one more amenable to simple scalar quantization.

### Codebook Generation and Quantization

Following rotation, the algorithm scales each group to match a learned Lloyd-Max codebook. The `_cq_codebook_np` function at line 13 generates normalized codebook vectors for the target bit-width. The quantization pipeline at lines 44-46 performs normalization, nearest-neighbor lookup, and de-rotation:

```python
unit = rot / jnp.maximum(norm, 1e-12)
quant = _cq_nearest(unit, cb)
deq = (quant * norm) @ H

```

This process maps continuous values to discrete 2-bit indices while preserving the angular relationships critical for model performance.

## Implementation Details in needle/model/quantize.py

The quantization system supports multiple bit-widths through a flexible API centered around three core functions.

### Supported Bit Widths

The constant `CQ_BITS` defined at line 199 specifies the available quantization levels:

```python
CQ_BITS = (2, 3, 4)

```

While the system supports 2, 3, and 4-bit modes, **CQ2-bit** remains the default configuration for Needle 2 deployments, offering the optimal balance between compression ratio and accuracy.

### Mixed-Precision Deployment

For heterogeneous model architectures, `cq_mixed_params` (line 21) enables per-layer bit-width selection through a mapping dictionary:

```python
def cq_mixed_params(params, names, bits_map, default_bits=2):
    b = _bits_for(names[i], bits_map, default_bits)
    # ... applies varying precision per layer

```

This allows sensitive layers to retain higher precision while compressing robust layers to the standard 2-bit format.

### Deployment Helper

The `deploy_quantize` function at line 74 returns a fully quantized parameter tree ready for inference:

```python
def deploy_quantize(params, config):
    return cq_quantize_params(params, _WEIGHT_BITS, _WEIGHT_GROUP)

```

By default, `_WEIGHT_BITS` initializes to 2, enforcing the CQ2-bit format unless explicitly overridden.

## Deploying CQ2-Bit Quantized Models

The following workflow demonstrates how to apply CQ2-bit quantization to a Needle 2 checkpoint and export it to a portable binary:

```python
import needle
from needle.model.quantize import configure_deploy, deploy_quantize

# Load a pretrained checkpoint (weights are downloaded automatically)

params = needle.model.load_weights("cactus-compute/needle2")

# Configure deployment for CQ2-bit (2-bit weights, 8-bit activations)

configure_deploy(act_bits=8, kv_bits=0)   # kv_bits=0 disables KV cache quantization

# Quantize the checkpoint for deployment

quantized_params, spec = deploy_quantize(params, config=None)
print(f"Deployment spec: {spec}")  # → "CQ W2"

# Export to a portable .cact archive

needle.build(
    checkpoint=params,
    out="my_needle.cact",
    quantized_params=quantized_params,
    bits=2                     # Forces CQ2-bit encoding

)

```

The resulting `my_needle.cact` file contains all quantized parameters in a single 14 MB binary that the Needle 2 runtime can load directly into approximately 28 MB of RAM, eliminating the need for external model files.

## Summary

- **CQ2-bit quantization** compresses 45M-parameter models to 14 MB by representing weights with 2 bits per value through learned codebook compression.
- **Hadamard rotation** in `cq_quantize` decorrelates weight dimensions before quantization, preserving model accuracy despite aggressive compression.
- The implementation supports **2, 3, and 4-bit modes** via the `CQ_BITS` tuple, with 2-bit as the default for maximum compression.
- **Mixed-precision deployment** via `cq_mixed_params` allows selective quantization of specific layers based on sensitivity.
- **Single-file deployment** produces `.cact` archives that contain all necessary weights for edge inference without additional dependencies.

## Frequently Asked Questions

### What compression ratio does CQ2-bit quantization achieve?

CQ2-bit quantization achieves approximately a 16:1 compression ratio compared to 32-bit floating point, packing a 45 million parameter model into roughly 14 MB. When loaded for inference, the model consumes approximately 28 MB of RAM, including overhead for the codebook and runtime buffers.

### How does Hadamard rotation improve quantization accuracy?

Hadamard rotation multiplies weight groups by a fixed Walsh-Hadamard matrix before quantization, decorrelating the dimensions and flattening the distribution. This preprocessing step allows a simple scalar quantizer to perform effectively on the rotated space, minimizing the angular error introduced by aggressive 2-bit discretization.

### Can I use bit widths other than 2 bits for quantization?

Yes. While CQ2-bit represents the default configuration, the system supports 3-bit and 4-bit quantization through the `CQ_BITS = (2, 3, 4)` constant. You can specify alternative bit widths by passing the `bits` parameter to `needle.build()` or configuring `_WEIGHT_BITS` directly before calling `deploy_quantize`.

### How do I export a model with CQ2-bit quantization for production?

Export a CQ2-bit quantized model by calling `deploy_quantize()` to generate quantized parameters, then pass these to `needle.build()` with `bits=2`. This produces a standalone `.cact` file containing all quantized weights and metadata required by the Needle 2 inference engine, suitable for deployment on resource-constrained devices.