How to Build a Tuned .cact File from a Needle Fine-Tuned Adapter

You can build a tuned .cact file by merging a LoRA adapter into the base Needle checkpoint using the needle build command, which internally calls merge_lora() to apply adapter weights, runs quantization via quantize(), and exports a self-contained engine file using save_cact().

The Needle repository provides a compact (~45M parameter) foundation model optimized for tool-calling and structured extraction, distributed as a single self-contained .cact engine file that runs on approximately 28 MiB of RAM. After fine-tuning the base model with LoRA adapters using needle/model/finetune.py, you must merge those weights and quantize the result to produce a deployment-ready .cact file. This guide explains the complete workflow from adapter training to final export, referencing the actual source implementation in the cactus-compute/needle repository.

Understanding the Needle Architecture

Before building a tuned engine, it helps to understand the Simple Attention Network (SAN) that powers Needle. Defined in needle/model/architecture.py, this architecture employs several specialized components to achieve high efficiency on limited hardware.

Core Components

The SAN implementation includes:

• Embedding & Scaling – A learned token embedding (nn.Embed) multiplied by √d for stable gradient flow, located at line 83 of architecture.py.

• Engram Memory – A learned n-gram KV store that augments standard attention with compressed long-range context, enabling a 256-token sliding window with KV sinks to bound memory usage (Engram class, line 81).

• Hadamard MLP – A Walsh-Hadamard transform that replaces the classic feed-forward network, dramatically reducing parameter count while providing non-linear mixing (HadamardMLP, line 87).

• Z-CRMSNorm – A custom layer normalization that stabilizes training across mixed-precision by normalizing via RMS and learning a per-channel scale (line 46).

• Contrastive & Confidence Heads – Probe-based heads producing embeddings for retrieval and calibrated confidence scores, powering tool retrieval and gated execution (ContrastiveHead at line 44 and ConfidenceHead at line 63).

All modules are implemented with Flax + JAX, allowing execution on CPUs, GPUs, and Apple Silicon without recompilation.

Fine-Tuning with LoRA

The fine-tuning pipeline lives in needle/model/finetune.py. This script loads the base checkpoint (auto-downloaded from Hugging Face if not provided locally), optionally synthesizes training data via OpenRouter, and trains a low-rank adapter.

Creating the Adapter

The LoRA adapter is instantiated with configurable rank and alpha parameters:

# Excerpt from finetune.py (line ~180)

adapter = LoRA(adapter_rank=args.lora_rank, alpha=args.lora_alpha)
params = adapter.init(rng, dummy_input)  # Base params remain frozen

The adapter is saved as a .pkl file (e.g., needle_lora.pkl) containing only the low-rank update matrices, keeping storage minimal.

Running Fine-Tuning via CLI

# Optional: Generate synthetic data

export OPENROUTER_API_KEY=sk-...
needle generate-data --tools my_tools.json --num-samples 500 --output data.jsonl

# Train the adapter

needle finetune data.jsonl --epochs 10 --lora-rank 16 --lora-alpha 32 --out my_lora.pkl

Building the Tuned .cact File

The build step, also implemented in finetune.py, converts your trained adapter into a standalone engine file. This process involves three critical operations: merging, quantization, and serialization.

The Build Pipeline

Inside build_main(), the following call chain executes:

# Simplified logic from finetune.py

merged = merge_lora(base_ckpt, lora_path)      # Applies LoRA weights to base

cact   = quantize(merged, bits=args.bits)      # CQ-2-bit or CQ-4-bit quantization

save_cact(cact, args.out)                     # Exports to single .cact file

• merge_lora() – Mathematically adds the low-rank adapter weights to the base checkpoint parameters, producing a full-rank model.
• quantize() – Applies per-layer quantization according to the checkpoint’s bit map (defaulting to 4-bit CQ) to reduce file size and memory footprint.
• save_cact() – Archives the quantized weights and metadata into a single .cact file via the export logic in needle/model/export.py.

CLI Build Command

needle build checkpoints/needle2.pkl \
       --lora my_lora.pkl \
       --out my_needle.cact \
       --bits 2    # Optional: 2-bit CQ for smaller size

The resulting my_needle.cact contains everything needed for inference—no external weight files or compilation steps required.

Practical Code Examples

Installation

pip install cactus-needle

For hardware acceleration:

• GPU: pip install "cactus-needle[gpu]"
• Apple Silicon: pip install "cactus-needle[metal]"

Complete Workflow Example

import needle

# 1. Define tools with type hints and docstrings

@needle.tool
def get_weather(city: str):
    "Return the current weather for a given city."
    return {"city": city, "temp_c": 27, "sky": "clear"}

# 2. Load the tuned .cact file

agent = needle.Needle(weights="my_needle.cact", tools=[get_weather])

# 3. Execute

response = agent.run("What’s the weather like in Lagos?")
print(response["results"])

# → [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

Structured Extraction

from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)
print(invoice.vendor, invoice.total)

# → Acme Corp 1200.0

Running the Playground UI

needle playground --weights my_needle.cact

# Access http://127.0.0.1:7860

The playground interface allows browser-based tool editing, query testing, and one-click fine-tuning triggers that regenerate .cact files automatically.

Summary

• Needle uses a Simple Attention Network with Engram Memory and Hadamard MLP to achieve ~45M parameters in a ~28 MiB footprint.
• Fine-tuning occurs via LoRA adapters stored as .pkl files, implemented in needle/model/finetune.py.
• Building a tuned .cact requires running merge_lora() to fuse the adapter, quantize() for compression (CQ-2-bit or CQ-4-bit), and save_cact() for export.
• The final .cact file is self-contained and loaded via needle.Needle(weights="path/to/file.cact").
• All operations are accessible via the needle CLI: finetune, build, and playground.

Frequently Asked Questions

What is a .cact file and why is it used?

A .cact file is a self-contained engine archive that bundles quantized model weights, metadata, and execution graphs into a single file. According to the Needle source code, this format allows the ~45M parameter model to run on devices with only ~28 MiB of RAM without requiring external dependency downloads or runtime compilation.

How does the LoRA merging process work internally?

The merging process implemented in needle/model/finetune.py calls merge_lora(base_ckpt, lora_path), which loads the frozen base parameters and the trained adapter, then mathematically combines them by adding the low-rank update matrices to the corresponding base weights. This produces a full-rank model ready for quantization, rather than performing adapter inference at runtime.

Can I adjust the quantization bit-width when building?

Yes. The needle build command accepts a --bits argument (typically 2 or 4) that controls whether the quantize() function applies CQ-2-bit or CQ-4-bit compression. Lower bit-widths produce smaller files but may slightly impact accuracy, while 4-bit offers a balance between size and precision for most tool-calling tasks.

How do I load a custom .cact file for inference?

Load the file by passing its path to the weights parameter of the Needle class: agent = needle.Needle(weights="my_needle.cact", tools=[...]). The engine validates the schema automatically and initializes the Simple Attention Network with the quantized weights contained in the archive.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}