# How Cactus Quants Achieves CQ2-Bit Compression for Needle 2

> Discover how Cactus Quants achieves CQ2-bit compression for Needle 2. Learn about block-wise Hadamard rotation, L2 normalization, and scalar quantization for efficient weight compression and fast inference.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-19

---

**Cactus Quants achieves CQ2-bit compression by applying a block-wise Hadamard rotation, per-group L2 normalization, and Lloyd-Max optimized scalar quantization to compress each weight to one of four levels, then packing the result for fast inference.**

Cactus Quants implements Compressed-Quant (CQ) as a deterministic, block-wise quantizer designed for low-bit inference in the Needle 2 framework. The CQ2-bit mode squeezes model weights to just two bits per element while preserving accuracy through a tight integration of optimal codebooks, orthogonal transforms, and per-group normalization. According to the `cactus-compute/needle` source code, the entire pipeline lives in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) and is orchestrated by a small set of core functions.

## Core CQ2-Bit Compression Pipeline in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)

The CQ quantizer processes weight matrices in three mathematical steps: codebook generation, Hadamard rotation, and scalar quantization with inverse rotation. A fourth step, `cq_quantize_params`, applies this block-wise routine across the model tree only to kernel and embedding tensors.

### Step 1 – Codebook Generation with `_cq_codebook_np`

For a requested bit-width *b*, the quantizer builds a static codebook of optimal scalar levels. When `bits = 2`, the function `_cq_codebook_np(bits, group_size)` returns a codebook containing four normalized levels derived from a Lloyd-Max Gaussian optimizer (`_lloyd_max_gaussian`) or, in ternary mode, from the predefined `_TERNARY_CB` constant. These four levels represent the only values a quantized weight can take in CQ2-bit mode.

### Step 2 – Hadamard Rotation via `_cq_hadamard_np`

Each weight matrix is split into column groups of size `group_size = 128`. The helper `_cq_hadamard_np` supplies a Hadamard matrix `H`, and `cq_quantize` performs the rotation `rot = groups @ H`. This orthogonal transform spreads information uniformly across dimensions, decorrelating the columns so that aggressive scalar quantization introduces less structured error.

### Step 3 – Scalar Quantization and Inverse Rotation

After rotation, each group vector is L2-normalized to produce a unit vector and a per-group scale (`norm`). The helper `_cq_nearest(unit, cb)` snaps every component to the nearest entry in the 2-bit codebook. The de-quantized approximation is then recovered with inverse rotation:

```python
deq = (_cq_nearest(unit, cb) * norm) @ H

```

The final tensor is sliced back to its original shape, preserving the approximate weight values at two bits of storage per element.

### Step 4 – Block-Level Model Quantization with `cq_quantize_params`

The top-level function `cq_quantize_params` traverses the model parameter tree and applies the above routine selectively to kernel and embedding tensors. For kernels where the last two axes are transposed according to the `_reduces_second_last` predicate, the function swaps axes before quantization and restores the original layout afterward. This ensures the compressed representation remains compatible with the forward pass.

## Why CQ2-Bit Quantization Retains High Fidelity

The CQ2-bit mode works well because of three specific design choices implemented in `cactus-compute/needle`:

- **Hadamard rotation** (`_cq_hadamard_np`) decorrelates matrix columns before quantization, enabling a tiny four-level codebook to capture most of the variance.
- **Per-group normalization** preserves energy locally, limiting quantization error to a small scale factor per 128-element block.
- **Group size of 128** aligns with GPU and CPU memory boundaries and allows the Hadamard transform to be computed efficiently without excessive overhead.

Together these ingredients give a compact CQ2-bit representation that can be packed and streamed to the inference engine with minimal runtime cost.

## Practical Code Examples

The following examples show how to quantize a trained model to CQ2-bit and prepare it for deployment.

### Quantizing Weights to CQ2-Bit

```python
import jax.numpy as jnp
from needle.model.quantize import cq_quantize_params, CQ_BITS

# params is a nested dict of JAX arrays from a trained Needle model

bits = 2          # CQ_BITS[0]

group_sz = 128    # default block size used by Cactus Quants

# Perform block-wise CQ quantization

quantized_params = cq_quantize_params(params, bits, group_sz)

```

### Deploying a Model with CQ2-Bit Weights

```python
from needle.model.export import export
from needle.model.quantize import deploy_quantize

# config may specify weight_bits, e.g., config.weight_bits = "default=2"

quant_params, spec = deploy_quantize(params, config)  # CQ2-bit internally

export(quant_params, spec, out_path="model.cq2")

```

## Key Source Files for CQ2-Bit Compression

Understanding the full CQ2-bit stack requires looking at three files in the `cactus-compute/needle` repository:

- **[`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)** – Contains `_cq_codebook_np`, `_cq_hadamard_np`, `_cq_nearest`, `cq_quantize`, and `cq_quantize_params`. This is the core of the CQ2-bit implementation.
- **[`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)** – Packs CQ tensors into a binary format for runtime loading and inference.
- **[`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)** – Connects the quantized weights to the forward pass via the `quant` flag.

The constant `CQ_BITS = (2, 3, 4)` at the bottom of [`quantize.py`](https://github.com/cactus-compute/needle/blob/main/quantize.py) explicitly declares the supported bit-widths, with `2` being the most aggressive compression tier.

## Summary

- Cactus Quants uses a **block-wise orthogonal transform** pipeline in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) to produce CQ2-bit weights.
- **`_cq_codebook_np`** generates a four-level codebook for 2-bit mode using Lloyd-Max optimization or a predefined ternary table.
- **`_cq_hadamard_np`** rotates 128-column groups to decorrelate weights before scalar quantization.
- **`_cq_nearest`** maps normalized unit vectors to the codebook, and inverse rotation restores approximate weights.
- **`cq_quantize_params`** applies the process across the model tree, handling transposed kernels automatically.
- The final packed format is written by [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) for efficient runtime inference.

## Frequently Asked Questions

### What is CQ2-bit compression in Needle 2?

CQ2-bit compression is a mode of the Compressed-Quant (CQ) system that stores each model weight using only two bits. It is implemented in `cactus-compute/needle` as a deterministic pipeline combining Hadamard rotation, per-group normalization, and four-level scalar quantization.

### How does the Hadamard rotation improve 2-bit quantization?

The Hadamard rotation, performed by `_cq_hadamard_np` and applied in `cq_quantize` as `groups @ H`, spreads weight information uniformly across dimensions. This decorrelation step prevents the tiny 2-bit codebook from missing critical variance, which dramatically reduces reconstruction error.

### What group size does Cactus Quants use for CQ2-bit mode?

The default and recommended group size is **128 columns**. This block size is chosen for efficient memory alignment on GPUs and CPUs and for fast computation of the Hadamard transform inside [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py).

### Where are CQ2-bit weights packed for runtime inference?

After quantization, `deploy_quantize` and [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) pack the CQ2-bit tensors into a binary stream. The resulting file can be loaded by the inference engine with the `quant=True` path in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), keeping dequantization overhead minimal.