# How to Fine-Tune Needle with LoRA and Build a Tuned Archive

> Learn to fine-tune Needle with LoRA. Generate a dataset, train a LoRA adapter, and export a merged .cact archive for efficient inference with cactus-compute/needle.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-04

---

**To fine-tune Needle with LoRA, you generate a dataset, train a LoRA adapter using `needle finetune`, and export a merged `.cact` archive with `needle export --lora` for efficient inference.**

Needle, the inference engine from the `cactus-compute/needle` repository, supports parameter-efficient fine-tuning through Low-Rank Adaptation (LoRA). This workflow allows you to specialize a frozen base model for specific tasks without touching the original weights, then bake the adaptations into a portable `.cact` archive. The entire process uses three CLI commands and keeps the tokenizer, confidence head, and runtime engine untouched.

## Prepare a JSONL Dataset

Needle expects training data in JSON Lines format, where each line contains a `"prompt"` and a `"completion"` field.

You can generate synthetic training data directly through the CLI:

```bash
needle generate --count 1000 --output data.jsonl

```

Alternatively, create the file manually with your own examples. The data capture logic and schema validation reside in [`needle/environments/data_capture.py`](https://github.com/cactus-compute/needle/blob/main/needle/environments/data_capture.py).

## Fine-Tune with LoRA

The `needle finetune` command trains a LoRA adapter on top of a frozen base checkpoint. According to the source code in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), the `finetune_local` function implements the training loop, while argument parsing lives in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py).

Key parameters for the fine-tuning command:

- **--lora-rank**: Rank of the LoRA adapters (default: `16`)
- **--lora-alpha**: Scaling factor for updates (default: `32.0`)
- **--checkpoint**: Base model identifier or path (auto-downloads from Hugging Face if needed)
- **--output**: Destination for the trained adapter weights (default: `adapter.pt`)
- **--epochs**: Number of training epochs (default: `3`)
- **--batch-size**: Mini-batch size (default: `8`)

Example command to fine-tune Needle with LoRA:

```bash
needle finetune \
  --checkpoint mistralai/Mistral-7B-v0.1 \
  --data data.jsonl \
  --lora-rank 16 \
  --lora-alpha 32.0 \
  --epochs 3 \
  --output my_adapter.pt

```

During this phase, the base model weights remain frozen; only the low-rank adapter matrices are updated. Progress displays through a text progress bar, and the resulting `.pt` file contains only the LoRA parameters.

## Build the Tuned .cact Archive

After training, merge the LoRA adapter into the base weights and export a unified archive. The export logic in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) handles the weight merging and serialization.

Run the export command with the `--lora` flag:

```bash
needle export \
  --checkpoint mistralai/Mistral-7B-v0.1 \
  --lora my_adapter.pt \
  --output tuned_model.cact

```

The resulting `tuned_model.cact` contains a single weight blob with the LoRA adaptations baked in. This eliminates runtime overhead during inference, as the engine loads one unified file rather than separate base and adapter weights.

## Verify and Deploy the Archive

Test the tuned model using the `run` command to confirm the adapter behavior:

```bash
needle run \
  --weights tuned_model.cact \
  --prompt "Explain the benefits of LoRA fine-tuning."

```

If the output reflects your domain-specific training, deploy the archive by moving `tuned_model.cact` into any compatible Needle runtime, such as the Playground server ([`needle/playground/server.py`](https://github.com/cactus-compute/needle/blob/main/needle/playground/server.py)). The archive requires no additional build steps and works in air-gapped environments because all weights are self-contained.

## Summary

- **Generate data** using `needle generate` or provide a manual JSONL file with `"prompt"` and `"completion"` fields.
- **Train the adapter** with `needle finetune`, specifying `--lora-rank` and `--lora-alpha` (defaults are 16 and 32.0).
- **Export the archive** using `needle export --lora <adapter>` to merge weights into a `.cact` file.
- **Deploy seamlessly** by loading the `.cact` archive into any Needle inference runtime.

## Frequently Asked Questions

### What file contains the core LoRA training implementation?

The `finetune_local` function in **[`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py)** contains the core training loop that freezes base weights and updates only the LoRA parameters. The CLI interface that exposes `--lora-rank` and `--lora-alpha` flags resides in **[`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)**.

### Does the base model get modified during LoRA fine-tuning?

No. The base checkpoint remains frozen throughout the process. Only the low-rank adapter matrices are trained and stored separately in a `.pt` file. The `needle export` command merges these adapters into the base weights only during the archive building phase.

### What are the default LoRA hyperparameters in Needle?

The defaults are **rank 16** (`--lora-rank 16`) and **alpha 32.0** (`--lora-alpha 32.0`). These values provide a balance between adaptation capacity and memory efficiency for most fine-tuning tasks.

### Can I use the tuned archive without the original base model?

Yes. The `.cact` archive produced by `needle export --lora` contains merged weights and is fully self-contained. You can load it directly with `needle run --weights tuned_model.cact` without requiring access to the original Hugging Face checkpoint or the separate adapter file.