# How to Fine-Tune a Needle Model with LoRA: Complete CLI Guide

> Learn how to fine-tune a Needle model with LoRA using the cactus-compute/needle CLI. Discover the command to train LoRA adapters efficiently, keeping base weights frozen.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-22

---

**The command to fine-tune a Needle model with LoRA is `needle finetune my_data.jsonl`, which trains Low-Rank Adaptation adapters on your JSONL dataset while keeping the base model weights frozen.**

Needle, the transformer library developed by cactus-compute, provides native support for **parameter-efficient fine-tuning** via the `finetune` sub-command. This implementation allows you to adapt large models to custom tasks without updating billions of base parameters, significantly reducing GPU memory requirements and storage costs.

## The Fine-Tune Command Syntax

The basic syntax follows this pattern:

```bash
needle finetune <dataset.jsonl> [OPTIONS]

```

According to the argument parser in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) (lines 31-50), the command supports the following key options:

- `--checkpoint`: Path to the base model checkpoint (default: `checkpoints/needle2.pkl`)
- `--lora-rank`: Dimension of the low-rank matrices, `r` (default: 16)
- `--lora-alpha`: Scaling parameter, `α` (default: 32)
- `--epochs`: Number of training epochs
- `--batch-size`: Training batch size
- `--lr`: Peak learning rate for the AdamW optimizer
- `--out`: Destination path for the saved adapter (default: `needle_lora.pkl`)

## Inside the LoRA Implementation

The fine-tuning workflow is implemented in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), which orchestrates the training pipeline through seven distinct stages:

### Loading the Base Checkpoint

The process begins by loading the frozen base parameters. In [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 106-108), the code resolves the checkpoint path:

```python
base_path = args.checkpoint or DEFAULT_BASE

```

If you do not specify `--checkpoint`, Needle defaults to `checkpoints/needle2.pkl`.

### Preparing the Training Data

Your JSONL file is processed through the `_encode` and `load_jsonl` functions (lines 215-252), which handle tokenization, sequence padding, and attention masking to create batched training examples compatible with the model's expected input format.

### Targeting Attention Projection Matrices

The `lora_target_paths` function (lines 54-62) automatically scans the model's parameter tree to identify attention projection kernels. By default, it targets standard transformer attention components such as `q_proj`, `k_proj`, `v_proj`, and `o_proj` weight matrices.

### Initializing Low-Rank Adapters

For each identified target matrix, the `init_lora` function (lines 66-83) initializes two small matrices: **A** (initialized with random values) and **B** (initialized to zero). These matrices have dimensions `(d, r)` and `(r, d)` respectively, where `r` is your configured `--lora-rank`. During training, only these low-rank matrices receive gradient updates.

### Scaling and Training Loop

The effective adaptation strength is controlled by the scaling factor `alpha / rank` (lines 130-132). With default values of `--lora-alpha 32` and `--lora-rank 16`, the scale factor is 2.0. The training loop uses **Optax's AdamW optimizer** with a cosine-decay learning rate schedule, strictly updating only the LoRA parameters while the original base weights remain frozen.

### Saving the Adapter

Upon completion, the adapter state—including the A and B matrices for all targets and the computed scale factor—is serialized to the path specified by `--out` (lines 89-99). This produces a compact `.pkl` file containing only the trainable parameters, typically orders of magnitude smaller than the full model checkpoint.

## Practical Code Examples

### Basic Fine-Tuning

Run fine-tuning with default LoRA settings (rank 16, alpha 32):

```bash
needle finetune my_data.jsonl --checkpoint checkpoints/needle2.pkl

```

### Advanced Configuration

Customize hyperparameters for larger rank and longer training:

```bash
needle finetune my_data.jsonl \
    --checkpoint checkpoints/needle2.pkl \
    --epochs 5 \
    --batch-size 32 \
    --lr 2e-4 \
    --lora-rank 32 \
    --lora-alpha 64 \
    --out tuned_adapter.pkl

```

### Merging the Adapter Programmatically

To combine your trained adapter with the base model for inference:

```python
from needle.model.finetune import merge_lora, load_checkpoint
import pickle
import jax.numpy as jnp

# Load base checkpoint

params, cfg = load_checkpoint("checkpoints/needle2.pkl")

# Load trained LoRA adapter

with open("tuned_adapter.pkl", "rb") as f:
    adapter = pickle.load(f)

# Reconstruct lora dictionary with proper key structure

lora = {
    tuple(k.split("/")): {
        "A": jnp.asarray(v["A"]), 
        "B": jnp.asarray(v["B"])
    }
    for k, v in adapter["lora"].items()
}

# Merge into base parameters

merged_params = merge_lora(params, lora, adapter["scale"])

# merged_params now contains the fine-tuned weights for inference

```

## Key Source Files

- **[`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py)**: Contains the complete LoRA implementation including `lora_target_paths` for matrix discovery, `init_lora` for adapter initialization, the Optax-based training loop, and `merge_lora` for combining adapters with base weights.
- **[`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)**: Defines the `finetune` sub-command interface and CLI argument parsing (lines 31-50), routing execution to `finetune_local`.

## Summary

- **Use `needle finetune <data.jsonl>`** to train LoRA adapters on custom datasets while preserving base model weights.
- **Key hyperparameters** `--lora-rank` (default 16) and `--lora-alpha` (default 32) control adapter capacity and scaling via the formula `alpha / rank`.
- **Target matrices** are automatically identified as attention projection layers (`q_proj`, `k_proj`, etc.) within the transformer blocks.
- **Training efficiency** is achieved through Optax AdamW with cosine decay, updating only the low-rank A and B matrices.
- **Output artifacts** are compact `.pkl` files that can be merged during the `build` step or loaded via `merge_lora` for inference.

## Frequently Asked Questions

### What is the exact command to fine-tune a Needle model with LoRA?

Run `needle finetune my_data.jsonl` followed by optional flags. According to [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), this invokes the `finetune_local` function in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py), which handles the complete LoRA training pipeline from data loading to adapter serialization.

### Which model layers does LoRA target in Needle?

The `lora_target_paths` function in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 54-62) automatically targets attention projection kernels including `q_proj`, `k_proj`, `v_proj`, and `o_proj` matrices. These weight matrices within the self-attention mechanism are identified by scanning the model's parameter tree for names matching the LoRA target list.

### How does Needle calculate the LoRA scaling factor?

As implemented in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 130-132), the scaling factor is computed as `alpha / rank` using the values provided via `--lora-alpha` and `--lora-rank`. With the default values of 32 and 16 respectively, the adapter outputs are scaled by 2.0 before being added to the base model activations.

### Can I merge the LoRA adapter back into the base checkpoint?

Yes. Use the `merge_lora` function available in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) to permanently combine the adapter weights with the base model parameters. Alternatively, you can include the adapter during the `needle build` step to produce a fully fine-tuned `.cact` archive ready for deployment.