# How to Build a Tuned .cact File from a Needle Fine-Tuned Adapter

> Build a tuned .cact file from a Needle fine-tuned adapter by merging LoRA weights. Learn how to use the needle build command to quantize and export your engine file.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-15

---

**You can build a tuned .cact file by merging a LoRA adapter into the base Needle checkpoint using the `needle build` command, which internally calls `merge_lora()` to apply adapter weights, runs quantization via `quantize()`, and exports a self-contained engine file using `save_cact()`.**

The **Needle** repository provides a compact (~45M parameter) foundation model optimized for tool-calling and structured extraction, distributed as a single self-contained `.cact` engine file that runs on approximately 28 MiB of RAM. After fine-tuning the base model with **LoRA** adapters using [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), you must merge those weights and quantize the result to produce a deployment-ready `.cact` file. This guide explains the complete workflow from adapter training to final export, referencing the actual source implementation in the `cactus-compute/needle` repository.

## Understanding the Needle Architecture

Before building a tuned engine, it helps to understand the **Simple Attention Network (SAN)** that powers Needle. Defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), this architecture employs several specialized components to achieve high efficiency on limited hardware.

### Core Components

The SAN implementation includes:

- **Embedding & Scaling** – A learned token embedding (`nn.Embed`) multiplied by √d for stable gradient flow, located at line 83 of [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py).

- **Engram Memory** – A learned n-gram KV store that augments standard attention with compressed long-range context, enabling a 256-token sliding window with KV sinks to bound memory usage (`Engram` class, line 81).

- **Hadamard MLP** – A *Walsh-Hadamard* transform that replaces the classic feed-forward network, dramatically reducing parameter count while providing non-linear mixing (`HadamardMLP`, line 87).

- **Z-CRMSNorm** – A custom layer normalization that stabilizes training across mixed-precision by normalizing via RMS and learning a per-channel scale (line 46).

- **Contrastive & Confidence Heads** – Probe-based heads producing embeddings for retrieval and calibrated confidence scores, powering tool retrieval and gated execution (`ContrastiveHead` at line 44 and `ConfidenceHead` at line 63).

All modules are implemented with **Flax + JAX**, allowing execution on CPUs, GPUs, and Apple Silicon without recompilation.

## Fine-Tuning with LoRA

The fine-tuning pipeline lives in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py). This script loads the base checkpoint (auto-downloaded from Hugging Face if not provided locally), optionally synthesizes training data via OpenRouter, and trains a low-rank adapter.

### Creating the Adapter

The LoRA adapter is instantiated with configurable rank and alpha parameters:

```python

# Excerpt from finetune.py (line ~180)

adapter = LoRA(adapter_rank=args.lora_rank, alpha=args.lora_alpha)
params = adapter.init(rng, dummy_input)  # Base params remain frozen

```

The adapter is saved as a **`.pkl`** file (e.g., `needle_lora.pkl`) containing only the low-rank update matrices, keeping storage minimal.

### Running Fine-Tuning via CLI

```bash

# Optional: Generate synthetic data

export OPENROUTER_API_KEY=sk-...
needle generate-data --tools my_tools.json --num-samples 500 --output data.jsonl

# Train the adapter

needle finetune data.jsonl --epochs 10 --lora-rank 16 --lora-alpha 32 --out my_lora.pkl

```

## Building the Tuned .cact File

The **build** step, also implemented in [`finetune.py`](https://github.com/cactus-compute/needle/blob/main/finetune.py), converts your trained adapter into a standalone engine file. This process involves three critical operations: merging, quantization, and serialization.

### The Build Pipeline

Inside `build_main()`, the following call chain executes:

```python

# Simplified logic from finetune.py

merged = merge_lora(base_ckpt, lora_path)      # Applies LoRA weights to base

cact   = quantize(merged, bits=args.bits)      # CQ-2-bit or CQ-4-bit quantization

save_cact(cact, args.out)                     # Exports to single .cact file

```

- **`merge_lora()`** – Mathematically adds the low-rank adapter weights to the base checkpoint parameters, producing a full-rank model.
- **`quantize()`** – Applies per-layer quantization according to the checkpoint’s bit map (defaulting to 4-bit CQ) to reduce file size and memory footprint.
- **`save_cact()`** – Archives the quantized weights and metadata into a single `.cact` file via the export logic in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py).

### CLI Build Command

```bash
needle build checkpoints/needle2.pkl \
       --lora my_lora.pkl \
       --out my_needle.cact \
       --bits 2    # Optional: 2-bit CQ for smaller size

```

The resulting `my_needle.cact` contains everything needed for inference—no external weight files or compilation steps required.

## Practical Code Examples

### Installation

```bash
pip install cactus-needle

```

For hardware acceleration:
- GPU: `pip install "cactus-needle[gpu]"`
- Apple Silicon: `pip install "cactus-needle[metal]"`

### Complete Workflow Example

```python
import needle

# 1. Define tools with type hints and docstrings

@needle.tool
def get_weather(city: str):
    "Return the current weather for a given city."
    return {"city": city, "temp_c": 27, "sky": "clear"}

# 2. Load the tuned .cact file

agent = needle.Needle(weights="my_needle.cact", tools=[get_weather])

# 3. Execute

response = agent.run("What’s the weather like in Lagos?")
print(response["results"])

# → [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

```

### Structured Extraction

```python
from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)
print(invoice.vendor, invoice.total)

# → Acme Corp 1200.0

```

### Running the Playground UI

```bash
needle playground --weights my_needle.cact

# Access http://127.0.0.1:7860

```

The playground interface allows browser-based tool editing, query testing, and one-click fine-tuning triggers that regenerate `.cact` files automatically.

## Summary

- **Needle** uses a **Simple Attention Network** with **Engram Memory** and **Hadamard MLP** to achieve ~45M parameters in a ~28 MiB footprint.
- Fine-tuning occurs via **LoRA** adapters stored as `.pkl` files, implemented in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py).
- Building a tuned `.cact` requires running **`merge_lora()`** to fuse the adapter, **`quantize()`** for compression (CQ-2-bit or CQ-4-bit), and **`save_cact()`** for export.
- The final `.cact` file is self-contained and loaded via `needle.Needle(weights="path/to/file.cact")`.
- All operations are accessible via the **`needle`** CLI: `finetune`, `build`, and `playground`.

## Frequently Asked Questions

### What is a .cact file and why is it used?

A `.cact` file is a self-contained engine archive that bundles quantized model weights, metadata, and execution graphs into a single file. According to the Needle source code, this format allows the ~45M parameter model to run on devices with only ~28 MiB of RAM without requiring external dependency downloads or runtime compilation.

### How does the LoRA merging process work internally?

The merging process implemented in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) calls `merge_lora(base_ckpt, lora_path)`, which loads the frozen base parameters and the trained adapter, then mathematically combines them by adding the low-rank update matrices to the corresponding base weights. This produces a full-rank model ready for quantization, rather than performing adapter inference at runtime.

### Can I adjust the quantization bit-width when building?

Yes. The `needle build` command accepts a `--bits` argument (typically `2` or `4`) that controls whether the `quantize()` function applies CQ-2-bit or CQ-4-bit compression. Lower bit-widths produce smaller files but may slightly impact accuracy, while 4-bit offers a balance between size and precision for most tool-calling tasks.

### How do I load a custom .cact file for inference?

Load the file by passing its path to the `weights` parameter of the `Needle` class: `agent = needle.Needle(weights="my_needle.cact", tools=[...])`. The engine validates the schema automatically and initializes the **Simple Attention Network** with the quantized weights contained in the archive.