# How Tensors Are Organized in the Needle .cact File Format

> Understand how tensors are organized in the Needle .cact file format. Explore its three-part binary structure: header, tensor directory, and aligned blob data containing various numerical formats.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: internals
- Published: 2026-08-30

---

**The `.cact` file format uses a three-part binary structure comprising a fixed-size header, a tensor directory with per-tensor metadata records, and aligned blob data containing IEEE-754 FP16/FP32 values, CQ-quantized packed weights, or RAW byte sequences.**

The `.cact` archive is the native serialization format for the [cactus-compute/needle](https://github.com/cactus-compute/needle) inference engine, designed for efficient storage and random-access loading of transformer model weights. All tensor organization logic is implemented in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py), which defines deterministic packing routines for quantized and full-precision parameters alongside embedded tokenizer vocabularies.

## The Archive Layout Structure

A `.cact` file follows a strict sequential layout that enables zero-copy slicing during model initialization:

```

| Header | Tensor Directory | Aligned Tensor Blobs |

```

This organization eliminates the need for external indexing structures while supporting mixed-precision storage and custom quantization schemes.

## Binary Header Specification

The archive begins with a fixed-size header block defined by the `_HDR_FMT` struct in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py). According to the `_pack_cact` function (lines 540‑560), this header encodes:

- **File tag** identifying the format version
- **Model configuration**: vocabulary size, hidden dimension, attention heads, and maximum sequence length
- **Quantization metadata**: code-book size for CQ tensors, plus `kv_window` and `kv_bits` parameters for key-value cache optimization

The header size remains constant regardless of model scale, allowing the loader to parse critical configuration values without scanning the entire file.

## Tensor Directory Records

Immediately following the header, the **tensor directory** contains one fixed-length record per tensor. Each record follows the `_REC_FMT` struct layout and is assembled during the directory construction phase in `_pack_cact` (lines 667‑72). The `_Tensor` class (lines 219‑226) serves as the internal container for these entries.

Each directory record stores the following metadata fields:

- **`dtype`**: Data type identifier (FP16 = 0, FP32 = 1, CQ = 2, RAW = 3)
- **`ndim`**: Number of tensor dimensions (up to 4)
- **`shape`**: Dimension sizes (zero-filled for unused axes)
- **`offset`**: Byte offset from file start to the tensor's blob
- **`nbytes`**: Length of the binary blob in bytes
- **`group`**: Group size for CQ-quantized tensors (0 otherwise)
- **`bits`**: Bit-width for quantization (0 for unquantized tensors)

The directory is built by iterating over the list returned from the `_tensors` helper function, which generates a `_Tensor` instance for every weight matrix, bias vector, and the special tokenizer blob.

## Tensor Blob Storage and Alignment

Following the directory, the actual tensor data is written sequentially with strict memory alignment. The `_pack_cact` function writes blobs in the final loop (lines 74‑78), applying a 64-byte alignment constraint via the `_align` helper to optimize SIMD access patterns.

The binary format varies by data type:

- **FP16/FP32**: Raw IEEE-754 floating-point representation
- **CQ (Compressed Quantized)**: Packed integer weights followed by FP16 normalization matrices (`packed.tobytes()` + `norms.tobytes()`)
- **RAW**: Unmodified byte sequences (used exclusively for the tokenizer)

Each blob is placed at the byte offset specified in its directory record, enabling the loader to memory-map individual tensors without parsing intermediate structures.

## Tokenizer Encoding

The tokenizer is embedded as a special RAW tensor named `"tokenizer"` (added in `_pack_cact` at lines 443‑46). Unlike weight matrices, this blob uses custom header structures (`_TK_HDR` and `_TK_REC`) to store:

- Total piece count in the vocabulary
- Special token IDs (control, unknown, byte markers)
- Per-piece metadata including UTF-8 byte sequences and type tags (control, unknown, user-defined, or normal)

This design ensures the model archive remains self-contained, eliminating external file dependencies during deployment.

## Loading and Parsing .cact Files

The `read_export` function (lines 96‑104) implements the inverse operation, reconstructing the model state from disk:

1. Parses the header to extract configuration and quantization parameters
2. Rebuilds the code-book for CQ tensors
3. Walks the tensor directory, slicing blobs according to recorded `offset` and `nbytes` values
4. Converts binary data back to NumPy arrays (or raw bytes for the tokenizer)

The function returns a tuple of `(metadata, tensors)`, where `tensors` is a list of NumPy arrays corresponding to the directory order.

## Working with .cact Files in Python

Export a checkpoint to the `.cact` format with quantization:

```python
from needle.model.export import write_export

# Parameters and config loaded from a checkpoint (e.g., .pkl file)

write_export(params, config, path="model.cact", bits=4, group=128)

```

Load tensors from an existing archive:

```python
from needle.model.export import read_export

metadata, tensors = read_export("model.cact")
print(f"Loaded {len(tensors)} tensors from {metadata['vocab_size']} vocab model")

# Access the embedding matrix (typically the first tensor)

embeddings = tensors[0]  # Shape: (vocab_size, d_model)

print(f"Embedding shape: {embeddings.shape}")

```

Decode the embedded tokenizer without external files:

```python
from needle.model.export import RefTokenizer

tok = RefTokenizer.from_cact("model.cact")
print(f"Vocabulary contains {len(tok.pieces)} pieces")

```

## Summary

- The `.cact` format organizes tensors into three contiguous regions: a fixed binary header, a metadata directory, and aligned data blobs.
- Each tensor record specifies data type, shape, quantization parameters, and file offset, supporting FP16, FP32, CQ-quantized, and RAW storage modes.
- The 64-byte alignment of tensor blobs ensures efficient memory mapping and SIMD compatibility during inference.
- Tokenizers are embedded as special RAW tensors with custom header structures, making archives fully self-contained.
- All serialization logic resides in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py), with `write_export` and `read_export` providing the primary public API.

## Frequently Asked Questions

### What is the maximum tensor dimensionality supported in .cact files?

The format supports up to four dimensions per tensor. The `shape` field in each directory record allocates four slots, with unused dimensions zero-filled. This accommodates standard transformer weight matrices while maintaining fixed-size directory entries.

### How does the format store quantized model weights?

CQ (Compressed Quantized) tensors store packed integer weights followed by FP16 normalization matrices. The directory entry records the `group` size and `bits` width, allowing the loader to reconstruct floating-point values using the code-book defined in the header.

### Where is the tokenizer vocabulary stored within the archive?

The tokenizer exists as a special RAW tensor named `"tokenizer"` with its own internal header structure (`_TK_HDR`/`_TK_REC`). It encodes vocabulary pieces as UTF-8 byte sequences with type tags, eliminating external vocabulary file dependencies.

### Why are tensor blobs aligned to 64-byte boundaries?

The `_align` function enforces 64-byte alignment to optimize cache line utilization and enable SIMD instruction sets during inference. This alignment occurs during export in `_pack_cact` and is respected during loading to prevent misaligned memory access.