# What Information Is Stored in the .cact Archive Header?

> Discover what information the .cact archive header stores, including head_dim, kv_window size, and engine version. Optimize your Needle inference runtime initialization.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: internals
- Published: 2026-08-30

---

**The .cact archive header stores a single `head_dim` value, a single `kv_window` size, and engine version metadata, allowing the Needle inference runtime to initialize transformer layers without per-layer configuration tables.**

The **.cact** format is a self-contained binary archive used by the **Cactus Compute Needle** engine to package trained transformer models. Unlike verbose metadata formats that store per-layer configurations, the .cact header is intentionally minimal, embedding only the essential dimensional parameters required for attention computation and KV-cache management.

## Core Header Fields in the .cact Format

The header defined in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) contains three critical pieces of metadata that precede the actual weight tensors in the archive.

### head_dim: Unified Attention Head Dimension

According to the export routine in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py), the header stores a single **`head_dim`** integer that represents the attention head dimension for the entire model. As noted in the source comments at lines 108–109, the "cact format stores a single head_dim; split attention dims" are normalized during export rather than stored per layer. This design choice reduces archive size and ensures consistent head dimensions across all transformer layers during inference.

### kv_window: Local/Global Attention Window Size

The header carries a single **`kv_window`** parameter that defines the sliding window size for the KV-cache. Lines 117–118 of [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) explicitly state that the "cact header carries a single kv_window; the local/global layer" implementation uses this value to determine how many recent tokens each attention layer can attend to. This parameter is crucial for memory-efficient inference on long contexts.

### Engine Version Compatibility

While not explicitly detailed in the inline comments, the `_pack_cact` function (implemented around line 340 in [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py)) implicitly includes version information in the header structure. This allows the `Needle` loader to validate archive compatibility before attempting to map weights into memory, preventing runtime errors from mismatched engine versions.

## Source Code Locations

Understanding where these fields are written and read provides insight into the archive lifecycle.

**Writing the header**: The [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) module handles serialization. The `_pack_cact` function writes the `head_dim` and `kv_window` integers into the file preamble before appending quantized weight tensors.

**Reading the header**: When initializing a model, the `Needle` class (defined in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)) invokes the loading routine—typically `Needle.from_cact` or the main constructor when passed a `.cact` file path. This loader parses the header bytes, validates the version, and uses `head_dim` and `kv_window` to configure the transformer architecture defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py).

## Practical Usage: Exporting and Loading .cact Files

The following examples demonstrate how the header fields are implicitly handled during export and load operations.

**Exporting a checkpoint to .cact format:**

```python

# The export routine writes head_dim and kv_window into the header automatically

import needle.model.export as export

export.main([
    "--checkpoint", "checkpoints/needle2.pkl",
    "--out", "model.cact"
])

```

**Loading a .cact archive:**

```python
from needle import Needle

# The engine reads the header (head_dim, kv_window) during initialization

agent = Needle(weights="model.cact", tools=[...])

```

In the loading example, the `Needle` class extracts the header metadata to initialize the underlying transformer layers with the correct attention dimensions and KV-cache window size before mapping the remaining archive bytes into model weights.

## Summary

- The **.cact** archive header is minimal by design, storing only essential metadata before the weight tensors.
- It contains a single **head_dim** value (attention head dimension) for all layers, as implemented in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py).
- It stores a single **kv_window** value that defines the local/global attention sliding window size.
- The header includes engine version information written by `_pack_cact` and validated by the `Needle` loader.
- These fields enable the inference engine to initialize architectures defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) without requiring external configuration files.

## Frequently Asked Questions

### Does the .cact header store per-layer attention dimensions?

No. According to the source comments in [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py), the cact format stores a single `head_dim` value that applies uniformly across all layers. The export routine normalizes or collapses per-layer dimensions into this single header field to keep the archive compact.

### What is the kv_window parameter used for?

The `kv_window` defines the size of the key-value cache sliding window for local/global attention mechanisms. As noted in [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) line 117, this header field tells the inference engine how many recent tokens each attention layer can attend to, which is critical for managing memory usage during long-context inference.

### How does Needle validate .cact archive compatibility?

The `_pack_cact` function in [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) embeds engine version metadata into the header structure. When `Needle.from_cact` or the main constructor loads a `.cact` file, it parses and validates this version information before mapping weights, ensuring that archives created by incompatible exporter versions are rejected early in the initialization process.

### Where is the header parsed when loading a model?

The header is parsed in the `Needle` class initialization logic, typically found in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py). This loader reads the initial bytes of the `.cact` file to extract `head_dim`, `kv_window`, and version fields, then uses these values to configure the transformer architecture before loading the weight data into memory.