How to Build a Tuned .cact Archive from a Base Model and LoRA Adapter

Run needle build <base>.pkl --lora <adapter>.pkl --out <tuned>.cact to merge a LoRA adapter into a base checkpoint and export a quantized, inference-ready .cact archive.

Needle's three-stage tuning workflow lets you customize large language models without full retraining, culminating in a single portable archive. This guide walks through building a tuned .cact archive from a base model and LoRA adapter using the official CLI commands from the cactus-compute/needle repository.

Fine-Tune a LoRA Adapter

The workflow begins by training a low-rank LoRA adapter on your dataset while keeping the base model weights frozen.

Run the needle finetune command on a JSONL dataset to generate an adapter file:

needle finetune data.jsonl \
    --epochs 10 \
    --lora-rank 16 \
    --lora-alpha 32 \
    --out adapter.pkl

• The command downloads the default base checkpoint automatically from Hugging Face if not specified.
• Training produces a *.pkl file containing the low-rank update matrices.
• Key hyperparameters include --lora-rank (matrix rank), --lora-alpha (scaling factor), --lr (learning rate), and --batch-size.

According to the source code in needle/cli.py, the finetune subcommand delegates to the training implementation in needle/model/finetune.py, which handles adapter serialization and checkpoint management.

Merge and Export the Tuned Archive

Once you have the base checkpoint and LoRA adapter, use the needle build command to merge the weights and package everything into a .cact archive.

needle build checkpoints/needle2.pkl \
    --lora adapter.pkl \
    --out tuned.cact

• --lora <adapter>.pkl loads the adapter and mathematically merges its low-rank updates into the base weights.
• --out <tuned>.cact specifies the destination file path.
• The resulting .cact archive contains all runtime assets: the engine binary, tokenizer, confidence head, and the merged quantized weights.

For aggressive compression, add --bits 2 to force 2-bit quantization across all layers (by default, the builder respects the checkpoint's per-layer bit map).

The CLI argument definitions for the build command are located at lines 138-166 in needle/cli.py, while the actual export logic that serializes the merged weights into the archive format lives in needle/model/export.py.

Publish the Tuned Archive (Optional)

To share the tuned model across machines or deploy to production, publish the .cact directly to Hugging Face:

export NEEDLE_HF_REPO=myorg/needle-tuned
needle build checkpoints/needle2.pkl \
    --lora adapter.pkl \
    --out tuned.cact \
    --upload

Other systems can then retrieve the archive using needle download myorg/needle-tuned/tuned.cact, eliminating manual file transfers.

Complete Workflow Example

The following pipeline demonstrates the full path from raw data to running inference:

# 1. Fine-tune the adapter

needle finetune data.jsonl --epochs 10 --lora-rank 16 --lora-alpha 32

# 2. Build the tuned .cact archive

needle build checkpoints/needle2.pkl \
    --lora checkpoints/needle_lora.pkl \
    --out my_needle.cact

# 3. (Optional) Upload to Hugging Face

export NEEDLE_HF_REPO=myorg/models
needle build checkpoints/needle2.pkl \
    --lora checkpoints/needle_lora.pkl \
    --out my_needle.cact \
    --upload

Load the archive directly into the inference engine without additional compilation:

import needle

agent = needle.Needle(
    weights="my_needle.cact",
    tools=[my_tool1, my_tool2]
)

response = agent.run("Dim the lights to 30% and lock the front door")
print(response["results"])

Summary

• Fine-tune first: Use needle finetune to train a LoRA adapter and save it as a .pkl file.
• Merge and build: Use needle build with --lora to fuse the adapter into the base weights and export a .cact archive.
• Quantization control: Add --bits 2 for uniform 2-bit quantization or omit it to preserve per-layer bit maps.
• Direct deployment: The .cact file is self-contained and loads directly into needle.Needle() for immediate inference.

Frequently Asked Questions

What file formats does Needle use during the tuning workflow?

Needle uses .pkl files for intermediate checkpoints and LoRA adapters, and .cact archives for final deployment. The .cact format is a unified container that bundles the merged weights, tokenizer, confidence head, and engine metadata into a single file optimized for the Needle runtime.

Can I adjust quantization settings when building the .cact archive?

Yes. By default, needle build respects the per-layer bit map stored in the base checkpoint. To override this and force uniform quantization, pass --bits 2 (or another bit width) to the build command. This is implemented in the builder logic within needle/cli.py and processed during weight packing in needle/model/export.py.

Do I need to manually merge the LoRA weights before building?

No. The needle build command handles the merging automatically when you provide the --lora argument. The system loads the base checkpoint and the adapter .pkl, applies the low-rank updates mathematically, and then quantizes the fused weights before packing them into the .cact archive. This eliminates manual weight surgery or external merging scripts.

How do I load a tuned .cact archive for inference?

Instantiate the Needle class with the weights parameter pointing to your .cact file: agent = needle.Needle(weights="tuned.cact", tools=[...]). The engine extracts all necessary components from the archive automatically, requiring no separate tokenizer or configuration files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}