# How to Configure CQ2-bit Quantization (2-bit vs 4-bit) When Building a .cact File in Needle

> Configure CQ2-bit quantization for .cact files in Needle. Learn to use the --bits flag for 2-bit or 4-bit compression, or --bits-map for per-layer control to optimize your models.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Use the `--bits` CLI flag with the `needle build` command to select 2-bit compression (`--bits 2`) or 4-bit compression (`--bits 4`, default), or supply a `--bits-map` for per-layer control.**

Needle's `.cact` export format supports **CQ (code-book) quantization** to reduce model size through low-precision weight compression. When building a `.cact` file from a trained checkpoint, you configure the quantization bit-width via command-line flags parsed in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) and implemented in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py).

---

## Understanding CQ2-bit Quantization in Needle

The CQ quantizer compresses model weights using learnable codebooks. The bit-width determines the number of centroids in the codebook:

- **2-bit quantization**: 4 centroids per codebook → maximum compression, lower precision
- **4-bit quantization**: 16 centroids per codebook → balanced compression and accuracy (default)

The default weight-bits value is defined as `_WEIGHT_BITS = 4` in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)【/cache/repos/github.com/cactus-compute/needle/main/needle/model/quantize.py†L49-L53】.

---

## Basic Usage: Setting Bit-Width with `--bits`

The `--bits` flag is parsed in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)【/cache/repos/github.com/cactus-compute/needle/main/needle/cli.py†L56-L57】 and forwarded to the export routine `build_export`.

### Default 4-bit Quantization

```bash

# 4-bit is the default; --bits 4 is optional

needle build checkpoints/needle2.pkl --out my_model.cact

```

### Explicit 2-bit Quantization

```bash

# Force 2-bit quantization for maximum compression

needle build checkpoints/needle2.pkl --bits 2 --out my_model_2bit.cact

```

---

## Advanced Usage: Per-Layer Control with `--bits-map`

For mixed-precision quantization, use `--bits-map` to specify different bit-widths for different tensors. The `parse_bits_map` function in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)【/cache/repos/github.com/cactus-compute/needle/main/needle/model/quantize.py†L301-L313】 parses this string format.

### Syntax and Example

The bits-map uses comma-separated `key=value` pairs:

```bash

# Default 2-bit, but layer0 (first attention) at 4-bit

needle build checkpoints/needle2.pkl \
    --bits-map "default=2,layer0=4" \
    --out my_model_mixed.cact

```

**Common patterns:**

- `"default=4"` — equivalent to omitting the flag
- `"default=2,layer0=4,layer1=4"` — aggressive compression with critical layers protected
- `"weight=2,embedding=4"` — tensor-type-based assignment (if supported by model naming)

---

## File Reference: Where Quantization Is Implemented

| File | Function | Role |
|------|----------|------|
| [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) | CLI argument parsing【/cache/repos/github.com/cactus-compute/needle/main/needle/cli.py†L56-L57】 | Parses `--bits` and `--bits-map` for `build` subcommand |
| [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) | `_WEIGHT_BITS = 4`, `parse_bits_map()`【/cache/repos/github.com/cactus-compute/needle/main/needle/model/quantize.py†L49-L53】【/cache/repos/github.com/cactus-compute/needle/main/needle/model/quantize.py†L301-L313】 | Defines defaults and parses per-tensor specifications |
| [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) | `build_export()` | Applies quantization settings when packing to `.cact` |
| [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) | `build_main()` | Entry point that forwards CLI options to export pipeline |

---

## Summary

- **Specify bit-width** with `--bits 2` or `--bits 4` (default) when running `needle build`
- **Use `--bits-map`** for fine-grained, per-layer quantization control via `parse_bits_map` in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)
- **4-bit is the default** as defined by `_WEIGHT_BITS = 4` in the quantizer source
- **2-bit reduces size further** but may degrade accuracy; use `--bits-map` to protect sensitive layers

---

## Frequently Asked Questions

### What is the difference between 2-bit and 4-bit CQ quantization in Needle?

2-bit quantization uses 4 codebook centroids to represent weights, while 4-bit uses 16 centroids. More bits mean higher precision and typically better model accuracy, but larger file sizes. The 4-bit default in [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)【/cache/repos/github.com/cactus-compute/needle/main/needle/model/quantize.py†L49-L53】 balances compression and quality.

### Can I mix 2-bit and 4-bit quantization in the same .cact file?

Yes. The `--bits-map` flag accepts a comma-separated string (e.g., `"default=2,layer0=4"`) that `parse_bits_map` processes【/cache/repos/github.com/cactus-compute/needle/main/needle/model/quantize.py†L301-L313】. This lets you compress most layers aggressively while keeping critical structures at higher precision.

### Where does Needle parse the --bits CLI argument?

The `needle build` command parses `--bits` and `--bits-map` in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)【/cache/repos/github.com/cactus-compute/needle/main/needle/cli.py†L56-L57】. These values propagate through `build_main` in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) to the export pipeline that generates the `.cact` archive.

### How do I verify which bit-width my .cact file uses?

The quantization configuration is embedded in the `.cact` archive metadata. While Needle does not expose a direct inspection command, you can infer the setting from the file size—2-bit models are roughly half the size of 4-bit equivalents for the same architecture—or by checking the build command history.