How to Export Tuned Checkpoints into .cact Archives in Needle

Needle exports fine-tuned models into self-contained .cact archives by optionally merging LoRA adapters, quantizing weights to INT4/INT3/INT2, and packing everything into a binary file with a 120-byte header.

The Needle inference engine stores complete models—weights, tokenizer, and metadata—in a single binary .cact file. This guide explains how to export tuned checkpoints, whether you're working with a raw base model or a LoRA-fine-tuned checkpoint, using both the CLI and Python API.

Understanding the Export Pipeline

The export process follows three logical stages implemented across needle/model/finetune.py and needle/model/export.py:

1. Load and optionally merge LoRA weights — build_main handles adapter fusion
2. Quantize and pack tensors — _pack_cact implements the binary format
3. Write the archive — write_export persists to disk

Each stage is optimized for deployment scenarios where model size and loading speed matter.

Step 1: Merge LoRA Adapters (Optional)

When exporting a fine-tuned checkpoint, you first need to combine the base parameters with the LoRA adapter. In needle/model/finetune.py, the build_main routine handles this merge:

# Source: needle/model/finetune.py (lines 13-19)

params = merge_lora(params, lora, adapter["scale"])

The merge_lora function:

• Deserializes the adapter with pickle.load
• Computes the low-rank update: W_merged = W_base + scale * (B @ A)
• Returns full-precision weights ready for quantization

Skip this step for base checkpoints without fine-tuning.

Step 2: Quantize and Pack Weights

The heavy lifting happens in needle/model/export.py. The write_export function calls _pack_cact, which:

• Derives model geometry via _geometry
• Constructs _Tensor objects for every layer, embedding, and probe head
• Packs quantized CQ matrices using custom LSB packing (_cq_pack for INT4/INT3/INT2)
• Stores FP16/FP32 tensors verbatim
• Builds a 120-byte header (_HDR_FMT) and nameless tensor directory (_REC_FMT)
• Aligns each tensor blob to 64 bytes (_align)

# Source: needle/model/export.py (lines 81-86)

buf, n = _pack_cact(params, config, bits, group, tokenizer, kv_window)

The quantization bits parameter controls compression:

• bits=4 — Default, ~4:1 compression (recommended)
• bits=2 — Maximum compression, ~8:1 (smaller models, slight quality tradeoff)

Step 3: Write the Archive to Disk

Finally, write_export writes the binary buffer and returns metadata:

# Source: needle/model/export.py (lines 87-93)

with open(path, "wb") as f:
    f.write(buf)

The function returns {"bytes": size, "tensors": count, "path": filepath} for verification.

CLI: Export Checkpoints with needle build

The needle build command wires all three steps together. It's defined in needle/cli.py and delegates to build_main in needle/model/finetune.py.

Export a base checkpoint (no LoRA):

needle build checkpoints/needle2.pkl --out my_needle.cact

Export a fine-tuned checkpoint with LoRA adapter:

needle build checkpoints/needle2.pkl \
      --lora checkpoints/needle_lora.pkl \
      --out my_needle_tuned.cact

The --lora flag triggers automatic merging before quantization and packing.

Python API: Programmatic Export

For custom workflows, call the export functions directly:

import needle
import pickle
import jax.numpy as jnp
from needle.model.export import write_export
from needle.model.run import load_checkpoint
from needle.model.tokenizer import get_tokenizer

# 1. Load base weights

params, config = load_checkpoint("checkpoints/needle2.pkl")[:2]

# 2. (Optional) Merge LoRA adapter

with open("checkpoints/needle_lora.pkl", "rb") as f:
    adapter = pickle.load(f)

params = needle.model.finetune.merge_lora(
    params,
    {tuple(k.split("/")): {"A": jnp.asarray(v["A"]), "B": jnp.asarray(v["B"])}
     for k, v in adapter["lora"].items()},
    adapter["scale"],
)

# 3. Export to .cact

out_path = "my_needle_tuned.cact"
info = write_export(
    params,
    config,
    out_path,
    bits=4,  # 4-bit CQ (default) — change to 2 for smaller models

    tokenizer=get_tokenizer(config.vocab_size),
    kv_window=needle.model.architecture.effective_kv_window(config),
)

print(f"Exported {info['bytes']/1e6:.2f} MiB, {info['tensors']} tensors → {info['path']}")

This pattern is useful when you need custom quantization settings or integration with training pipelines.

Key Source Files

File	Purpose	Direct Link
needle/model/export.py	.cact format implementation, _pack_cact, write_export	source
needle/model/finetune.py	build_main, merge_lora, CLI glue	source
needle/cli.py	needle build command parsing	source
needle/model/run.py	load_checkpoint for export pipeline	source
needle/model/tokenizer.py	Tokenizer blob embedding	source

Summary

• .cact archives bundle weights, tokenizer, and metadata in one binary file for Needle inference
• LoRA merging happens in merge_lora (needle/model/finetune.py) before quantization
• Quantization and packing use _pack_cact (needle/model/export.py) with custom bit packing for CQ matrices
• CLI workflow: needle build checkpoint.pkl --lora adapter.pkl --out model.cact
• Python API: load_checkpoint → merge_lora (optional) → write_export

Frequently Asked Questions

What quantization formats does .cact support?

Needle supports INT4, INT3, and INT2 CQ (compressed quantization) matrices, plus FP16 and FP32 for non-quantized tensors. The bits parameter in write_export selects the compression level—4-bit is the default balance of size and accuracy.

Can I export without merging LoRA weights?

Yes. Omit the --lora CLI flag or skip the merge_lora call in Python. The export will contain only the base checkpoint weights, which is useful when the LoRA adapter is loaded dynamically at runtime.

Why is tensor alignment set to 64 bytes?

The _align function ensures 64-byte alignment for SIMD efficiency during loading and inference. This is hardcoded in needle/model/export.py and matches the Needle runtime's memory access patterns.

How do I verify a .cact export succeeded?

write_export returns a dictionary with bytes (file size), tensors (count), and path. Additionally, you can inspect the archive with needle inspect path/to/model.cact to verify header integrity and tensor directory structure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}