# How to Export a Needle 2 Model with 2‑Bit Quantization: A Complete Guide

> Export your Needle 2 model with 2-bit quantization using the CLI or Python API. This guide details the process for efficient model deployment.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-29

---

**You can export a Needle 2 model with 2‑bit quantization by passing `--bits 2` to the `needle build` CLI or setting `bits=2` in the `write_export` Python API, which triggers the Cactus‑Quants (CQ) packing pipeline in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py).**

Needle 2 stores model weights in a custom binary format called **cact**, utilizing the **Cactus‑Quants (CQ)** system to compress matrices into 2‑, 3‑, or 4‑bit representations. When you export a Needle 2 model with 2‑bit quantization, you significantly reduce memory footprint while maintaining full inference compatibility, as the runtime engine automatically detects the bit‑width from the file header. This guide covers both command‑line and programmatic methods to generate 2‑bit quantized `.cact` files.

## Understanding Needle 2’s Cactus‑Quants (CQ) System

Needle 2 implements quantization through the `CB_BITS` constant in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) (lines 90‑92), which defines the supported bit widths for the CQ format. The quantizer supports 2‑, 3‑, and 4‑bit representations, with 2‑bit offering the highest compression ratio for deployment scenarios where model size is critical.

### The CB_BITS Constant and Supported Bit Widths

In the source code, the `CB_BITS` tuple explicitly lists the valid quantization levels. When you request 2‑bit export, the system validates your selection against this constant before proceeding to the packing phase.

## CLI Method: Using needle build with --bits 2

The fastest way to export a Needle 2 model with 2‑bit quantization is through the command‑line interface. The `main()` function in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) (lines 41‑58) parses the `--bits` argument and forwards it to `write_export`.

### Basic 2‑Bit Export

To convert a base checkpoint to 2‑bit format:

```bash
needle build checkpoints/needle2.pkl \
    --out my_needle_2bit.cact \
    --bits 2

```

The `--bits 2` flag forces the export pipeline to use 2‑bit codebooks during quantization. As documented in the README (lines 44‑46), this produces a `.cact` file containing LSB‑first packed indices and the corresponding 2‑bit codebook metadata.

### Exporting with LoRA Fine‑Tuning

If you have applied LoRA (Low‑Rank Adaptation) fine‑tuning, merge the adapters during export:

```bash
needle build checkpoints/needle2.pkl \
    --lora checkpoints/needle_lora.pkl \
    --out my_needle_2bit.cact \
    --bits 2

```

This combines the base model parameters with LoRA weights before applying 2‑bit quantization, ensuring the adapted model is compressed and exported as a single `.cact` artifact.

## Programmatic Method: The write_export Python API

For custom workflows, import `write_export` from `needle.model.export` and invoke it with `bits=2`. This method gives you explicit control over the quantization group size and tokenizer configuration.

```python
import needle
from needle.model.export import write_export

# Load checkpoint (params, config) – see needle.model.run.load_checkpoint

params, config, _ = needle.model.run.load_checkpoint("checkpoints/needle2.pkl")

# Export with 2‑bit CQ quantization

info = write_export(
    params,
    config,
    path="my_needle_2bit.cact",
    bits=2,                     # Forces 2‑bit quantization

    group=128,                  # Default group size for CQ packing

    tokenizer=needle.model.run.get_tokenizer(config.vocab_size),
)
print(f"Exported {info['path']} ({info['bytes']/1e6:.2f} MB, {info['tensors']} tensors)")

```

The `bits` parameter reaches `write_export` (lines 87‑90) and propagates through to `_pack_cact` (lines 81‑85), initiating the low‑bit packing pipeline.

## How 2‑Bit Quantization Works Under the Hood

When `bits=2` is specified, the export pipeline follows a specific call chain in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py): `write_export` → `_pack_cact` → `_q` → `_cq_pack`.

### The _cq_pack and _cq_codebook_np Functions

The `_cq_pack` helper selects the appropriate codebook using `_cq_codebook_np(bits, group)` (lines 84‑89). For 2‑bit quantization, this generates a 2‑bit codebook and packs the weight indices using an LSB‑first bit scheme. The packed blobs are then prepared for file serialization.

### Header Structure and Runtime Detection

During export, the `write_export` function constructs a header that records the quantization parameters. Specifically, the tensor directory stores the `bits` field (lines 33‑38), which the Needle runtime reads to automatically configure the dequantizer. Because the header explicitly declares the 2‑bit format, no additional runtime configuration is required to execute the model.

## Key Source Files for 2‑Bit Export

Understanding these files helps when debugging or extending the quantization pipeline:

- **[`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)**: Contains the core export logic, CQ packing implementation (`_cq_pack`), header construction, and CLI entry point (`main()`). This is where the `--bits` flag is parsed and the 2‑bit codebook is selected.
- **[`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py)**: Provides helper functions for CQ codebook generation, Hadamard transforms, and bit‑width handling used during the packing process.
- **[`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)**: Defines the model configuration (attention heads, engram settings, KV window size) embedded in the `.cact` header.
- **[`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md)**: User‑level documentation covering the `--bits 2` flag for smaller model exports.
- **[`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md)**: Describes the complete workflow from data synthesis through LoRA fine‑tuning to final quantized export.

## Summary

- Needle 2 uses **Cactus‑Quants (CQ)** to support 2‑, 3‑, and 4‑bit weight quantization via the `CB_BITS` constant in [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py).
- Use the CLI flag **`--bits 2`** with `needle build` to export 2‑bit models from the command line, compatible with both base and LoRA‑merged checkpoints.
- Call **`write_export(bits=2)`** in Python for programmatic export, allowing custom group sizes and tokenizer integration.
- The quantization pipeline invokes **`_cq_codebook_np(bits, group)`** and packs indices LSB‑first when 2‑bit mode is active.
- The exported `.cact` file header stores the bit‑width in the tensor directory, enabling **automatic runtime detection** without manual configuration.

## Frequently Asked Questions

### What bit widths does Needle 2 support for quantization?

Needle 2 supports 2‑bit, 3‑bit, and 4‑bit quantization through the Cactus‑Quants system. The `CB_BITS` constant in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) (lines 90‑92) explicitly defines these supported values, with 2‑bit providing the maximum compression ratio.

### Can I export a LoRA‑fine‑tuned model with 2‑bit quantization?

Yes. Pass both the base checkpoint and LoRA weights to `needle build`, then specify `--bits 2`. The CLI merges the adapters before quantization, outputting a single compressed `.cact` file that contains the fine‑tuned parameters in 2‑bit format.

### Does the Needle runtime require special configuration to load 2‑bit models?

No. The export process stores the bit‑width in the `.cact` file header (specifically in the tensor directory’s `bits` field). When you load the model, the engine reads this metadata and automatically initializes the appropriate dequantization kernels for 2‑bit inference.

### How do I verify that my export actually used 2‑bit quantization?

The `write_export` function returns a dictionary containing export statistics. Check the file size or inspect the `info` return value; programmatically, you can also examine the `.cact` header structure, which explicitly records `bits: 2` in the tensor metadata stored by the export logic in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) (lines 33‑38).