# How to Merge LoRA Adapters and Build a Tuned .cact File

> Merge LoRA adapters and build a tuned .cact file with needle. Use finetune_local to train adapters, then build_main to merge and export quantized binary parameters.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-10

---

**To merge LoRA adapters and build a tuned .cact file, use `finetune_local` in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) to train a low-rank adapter on your dataset, then run `build_main` to merge the adapter weights into the base model and export the combined parameters to a quantized .cact binary using `write_export`.**

The cactus-compute/needle repository implements a lightweight JAX-based inference engine that supports parameter-efficient fine-tuning through Low-Rank Adaptation (LoRA). When you merge LoRA adapters and build a tuned .cact file, you preserve the original base checkpoint while creating a standalone, compressed binary that contains both the model weights and fine-tuned knowledge, ready for immediate inference without additional dependencies.

## Step 1 – Train a LoRA Adapter with `finetune_local`

The first phase involves creating a low-rank adapter that learns task-specific updates without modifying the base model parameters. In [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), the `finetune_local` function orchestrates this process by loading the base checkpoint, initializing LoRA matrices for target weight groups, and executing a JAX training loop.

The function targets attention projection layers—including `q_proj`, `k_proj`, `v_proj`, `gate_proj`, and `out_proj`—and constructs "A" and "B" matrices for each. According to the source code around lines 67–84, the adapter initialization creates these low-rank decomposition matrices with configurable rank and alpha scaling. The training loop then runs for the specified number of epochs, optimizing only the adapter parameters while keeping the base weights frozen.

To train an adapter, prepare a JSONL dataset and invoke the training pipeline:

```python
from needle.model.finetune import finetune_local, argparse

parser = argparse.ArgumentParser()
parser.add_argument("--jsonl_path", required=True)
parser.add_argument("--checkpoint_dir", default="ckpt")
parser.add_argument("--lora_rank", type=int, default=8)
parser.add_argument("--lora_alpha", type=float, default=16)
parser.add_argument("--epochs", type=int, default=3)
parser.add_argument("--batch_size", type=int, default=32)
args = parser.parse_args([
    "--jsonl_path", "needle_data.jsonl",
    "--checkpoint_dir", "ckpt",
    "--lora_rank", "8",
    "--lora_alpha", "16",
    "--epochs", "3",
    "--batch_size", "32",
])

finetune_local(args)

```

This script loads the default base checkpoint from `checkpoints/needle2.pkl` (via `load_checkpoint` in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py)), initializes LoRA adapters with rank 8 and alpha 16, and serializes the trained adapter to `ckpt/needle_lora.pkl` using pickle. The output `.pkl` file contains only the low-rank matrices, keeping the storage footprint minimal.

## Step 2 – Merge the Adapter and Export to .cact

Once you have a trained adapter, the `build_main` function in [`finetune.py`](https://github.com/cactus-compute/needle/blob/main/finetune.py) handles both merging and export. This function loads the base checkpoint and the adapter pickle file, applies the low-rank updates to reconstruct the fine-tuned weights, and writes the result to the compact `.cact` format.

The merging process occurs in three stages:

1. **Load components**: `build_main` restores the base weights from the checkpoint path and deserializes the LoRA adapter from the `.pkl` file (lines 12–18).
2. **Apply LoRA updates**: The `merge_lora` function combines the base weights with the scaled low-rank matrices (B×A×alpha/rank), producing updated parameters that incorporate the fine-tuned knowledge.
3. **Quantize and export**: `write_export` from [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) serializes the merged weights, configuration, tokenizer, and KV-window metadata into a single binary, applying 4-bit quantization by default to minimize file size.

Execute the build process with the following configuration:

```python
from needle.model.finetune import build_main, argparse

parser = argparse.ArgumentParser()
parser.add_argument("--checkpoint", default="checkpoints/needle2.pkl")
parser.add_argument("--lora", required=True)
parser.add_argument("--out", default="needle_finetuned.cact")
parser.add_argument("--bits", default="4")
args = parser.parse_args([
    "--checkpoint", "checkpoints/needle2.pkl",
    "--lora", "ckpt/needle_lora.pkl",
    "--out", "needle_finetuned.cact",
    "--bits", "4",
])

build_main(args)

```

The `--bits` parameter controls quantization precision; the default 4-bit format reduces storage requirements while maintaining inference quality. After completion, `build_main` reports the final file size and tensor count (lines 32–34), confirming that the `.cact` binary contains all necessary components for standalone execution.

## Step 3 – Load the Tuned Binary for Inference

The resulting `.cact` file is self-contained and requires no external checkpoint or adapter files. The `Needle` class in the main package automatically restores the tokenizer, model weights, and KV-cache configuration from this binary.

To run inference with your fine-tuned model:

```python
from needle import Needle

model = Needle(
    weights="needle_finetuned.cact",
    tools=[...]  # JSON schema definitions for tool use

)

result = model("What are the top-3 most recent issues in the repo?")
print(result)

```

Because the `.cact` format bundles the tokenizer and architecture configuration alongside the weights, the initialization process requires only the single binary path. This design eliminates dependency management issues and ensures reproducible inference across different environments.

## Summary

- **LoRA training** uses `finetune_local` in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) to create low-rank adapter matrices for attention projections, storing the result in a pickle file.
- **Merging** occurs in `build_main`, which calls `merge_lora` to combine adapter updates with base weights before exporting.
- **Export** via `write_export` in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) produces a quantized `.cact` binary containing weights, tokenizer, and metadata.
- The final `.cact` file operates as a standalone artifact, loaded directly by the `Needle` class without requiring the original checkpoint or separate adapter files.

## Frequently Asked Questions

### What is the difference between a .pkl adapter and a .cact file?

The `.pkl` file contains only the LoRA adapter parameters—the low-rank A and B matrices trained during fine-tuning. It requires the base checkpoint to function and is used as an intermediate artifact. The `.cact` file is a merged, self-contained binary that includes the full model weights (base + merged adapter), tokenizer vocabulary, and configuration metadata, making it suitable for deployment and inference without external dependencies.

### Can I merge multiple LoRA adapters into a single .cact file?

The current `build_main` implementation in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) is designed to merge one adapter at a time with the base weights. To combine multiple adapters, you would need to sequentially load and merge them before the final export, or modify the `merge_lora` logic to accept multiple adapter paths. Each merge operation produces a new weight set that could theoretically serve as the base for subsequent adapter merges.

### What quantization options are available when building .cact files?

The `write_export` function supports configurable bit-width quantization via the `--bits` parameter in `build_main`. The default configuration uses 4-bit quantization to balance model size and inference accuracy. While the codebase implements 4-bit packing by default, the export architecture in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) can be extended to support alternative bit depths by modifying the quantization routine passed to `write_export`.

### Do I need the original base checkpoint to run the merged .cact file?

No. Once you have merged the LoRA adapter and exported the `.cact` binary, the original base checkpoint and adapter `.pkl` are no longer required. The `.cact` format embeds all necessary tensors, tokenizer data, and KV-window specifications, allowing the `Needle` class to initialize directly from the single binary file for inference.