# What Is the .cact Format and How Does the Export Process Work in Needle?

> Discover the .cact format, Needle's efficient binary model representation. Learn how the export process works and understand its role in the C++ inference engine for fast loading.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: internals
- Published: 2026-08-18

---

**The .cact file is Needle’s compact binary representation of a trained transformer model, packing all weights, Cactus-Quant metadata, and the tokenizer into a single memory-mapped blob that the C++ inference engine loads without parsing.**

Needle (from the `cactus-compute/needle` repository) stores trained models in the `.cact` format to enable instant loading and inference. Unlike JSON or pickle-based checkpoints, this binary archive embeds quantization codebooks, tensor geometry, and the tokenizer into a deterministic layout that the runtime can mmap directly.

## Architecture of the .cact Format

The `.cact` archive is structured as a contiguous byte stream with seven distinct regions. According to the source code in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py), the layout enables single-pass streaming without index files.

### Header (120 bytes)

The first 120 bytes encode the model’s architecture and quantization settings. In [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) lines 19‑31, the header consists of 30 `uint32` fields plus one `float32` (`rope_theta`) that describe vocabulary size, hidden dimensions, number of attention heads, and the codebook length.

### Codebooks

Following the header, the file stores concatenated Lloyd-Max codebooks used for **Cactus-Quant (CQ)** compression. The header field `codebook_len` specifies the total size of this region. These codebooks support INT4, INT3, and INT2 quantization as defined by `CB_BITS = (2, 3, 4)` in lines 25‑30 of [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py).

### Tensor Directory

A nameless directory of `REC_SIZE`‑byte records begins after the codebooks. Each record, detailed in lines 33‑38 of [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py), stores a tensor’s `dtype`, number of dimensions (`ndim`), shape array, byte offset, size in bytes, group size, and bit-width. This lets the runtime locate any tensor via fixed-size arithmetic without string lookups.

### Tensor Order

Tensors appear in a hard-coded canonical sequence so the engine can stream them sequentially. As implemented in lines 38‑44 of [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py), the order is:

- Embedding weights
- Per-layer tensors (input norms, Q/K/V projections, output projections, post-attention norms)
- Multi-head-cache blocks
- Engram tables
- Final normalization weights
- Optional probe heads
- Raw tokenizer blob

### Probe Heads

If the checkpoint contains LoRA-style adapters or auxiliary classifiers, `_head_tensors` (lines 44‑50) appends contrastive and confidence heads after `final_norm`. The directory records how many probe heads exist, followed by their projection and bias tensors.

### Quantization Layout

CQ-quantized tensors store packed indices LSB-first, accompanied by per-group L2 norms. The format supports ternary (5-bit record) fallback and the three standard bit-widths. Lines 51‑65 of [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) handle the bit-packing logic that converts FP16 weight matrices into these compressed representations.

### Tokenizer Blob

At the very end of the file, a raw `RAW` tensor contains a self-contained SentencePiece tokenizer. This blob stores piece counts, scores, types, and UTF-8 strings. The runtime reconstructs a `RefTokenizer` from this section without external vocabulary files, as shown in lines 68‑74 of [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py).

## The Needle Export Process

The export pipeline transforms a PyTorch or Flax checkpoint into a `.cact` archive through eight distinct stages. The entry point `write_export` in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) orchestrates this conversion.

**1. Load checkpoint** – The `main()` function calls `load_checkpoint` from [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) (lines 37‑43) to deserialize weights into NumPy arrays and a `TransformerConfig` object.

**2. Determine geometry** – `_geometry` validates head dimensions, engram settings, and KV-cache parameters, raising exceptions for unsupported configurations (lines 102‑119).

**3. Build tensor list** – `_tensors` creates a mixed-precision list: FP16 tensors (`_fp16`) for biases and layer norms, and CQ tensors (`_q`) for weight matrices. It packs each matrix using the specified bit-width and group size (lines 40‑64).

**4. Add optional heads** – `_head_tensors` (lines 92‑107) appends probe-head tensors if the model includes contrastive or confidence adapters.

**5. Pack tokenizer** – `_tokenizer_blob` (lines 17‑35) serializes the SentencePiece tokenizer into the raw blob format required by the runtime.

**6. Assemble header** – `_pack_cact` constructs the 120-byte header, appends the concatenated codebooks, and calculates each tensor’s byte offset aligned to 64-byte boundaries (lines 53‑61).

**7. Write directory and data** – The export logic writes the `REC_SIZE` directory records after the header, followed by zero-padding to each tensor’s offset and then the actual tensor blobs (lines 67‑78).

**8. Emit file** – `write_export` flushes the final `bytes` object to disk (default `needle.cact`) and returns file metadata (lines 86‑93).

### Exporting via CLI

You can generate a `.cact` file directly from the command line:

```bash

# Export a checkpoint to a tuned .cact archive

needle build checkpoints/needle2.pkl --out tuned.cact

```

### Programmatic Export

For custom pipelines, import the exporter directly:

```python
from needle.model.export import write_export
from needle.model.run import load_checkpoint
from needle.model.quantize import parse_bits_map

# 1. Load checkpoint into NumPy arrays

params, config, _ = load_checkpoint("checkpoints/needle2.pkl", return_run=True)

# 2. Configure quantization (e.g., 4-bit INT4)

bits = parse_bits_map("4")

# 3. Write the archive

info = write_export(
    params=params,
    config=config,
    path="my_model.cact",
    bits=bits,
    group=128,
    tokenizer=None,  # loads default automatically

)

print(f"Wrote {info['path']} ({info['bytes']/1e6:.2f} MB, {info['tensors']} tensors)")

```

### Reading a .cact File

To inspect or validate an archive:

```python
from needle.model.export import read_export

metadata, tensors = read_export("my_model.cact")
print(metadata)      # Architecture, kv_window, quantization settings

print(len(tensors))  # Total tensor count including tokenizer

```

## Summary

- The **.cact format** is a self-contained binary archive for Needle transformer models, embedding weights, quantization codebooks, and tokenizers.
- The **120-byte header** in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) encodes all architecture geometry and quantization parameters.
- **Cactus-Quant (CQ)** compression supports INT4, INT3, and INT2 via Lloyd-Max codebooks stored contiguously after the header.
- The **tensor directory** uses fixed-size records (`REC_SIZE`) to enable O(1) tensor lookup without name hashing.
- The **export process** (`write_export`) loads checkpoints via [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py), validates geometry, packs tensors, and aligns all data to 64-byte boundaries for efficient memory mapping.

## Frequently Asked Questions

### What quantization bit-widths does the .cact format support?

The format supports 2-bit, 3-bit, and 4-bit integer quantization through **Cactus-Quant (CQ)**, with configurable group sizes for per-block scaling. It also includes a 5-bit ternary fallback for special cases, as defined by `CB_BITS` in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py).

### How does the Needle runtime locate specific tensors without names?

The runtime uses the **tensor directory** stored immediately after the header. Each tensor occupies a fixed-size record (`REC_SIZE` bytes) containing its byte offset, dimensions, and data type. Because tensors follow a hard-coded canonical order (embedding → layers → cache → tokenizer), the engine can compute addresses directly without parsing names.

### Can I export a model programmatically without using the CLI?

Yes. Import `write_export` from `needle.model.export` and `load_checkpoint` from `needle.model.run`. Pass the loaded parameters and configuration to `write_export` along with your desired bit-width and group size. This approach is essential for integrating Needle exports into custom training pipelines.

### Why does the .cact file include the tokenizer as a raw blob?

Embedding the tokenizer eliminates external file dependencies. The final section of the archive stores a serialized SentencePiece vocabulary (piece count, scores, types, and UTF-8 strings). This allows the C++ inference engine to reconstruct a `RefTokenizer` directly from the mmap’d file without reading secondary assets from disk.