# How to Fine-Tune Needle 2 with LoRA Using the CLI and Key Parameters Like `--lora-rank`

> Learn to fine-tune Needle 2 with LoRA using the CLI. Explore key parameters like --lora-rank and --lora-alpha for efficient model adaptation without altering base weights.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Use `needle finetune` with `--lora-rank` to set the low-rank dimension (default: 16), `--lora-alpha` to control scaling, and `--lora` to load existing adapters for merging—enabling parameter-efficient fine-tuning without modifying base model weights.**

The **Needle 2** framework from **cactus-compute/needle** provides a streamlined command-line interface for LoRA (Low-Rank Adaptation) fine-tuning. This approach dramatically reduces memory and compute requirements by training only small adapter matrices while keeping the base model frozen. The CLI exposes precise control over rank, scaling, and target layer selection through well-documented flags.

## Understanding LoRA Parameters in Needle 2

Fine-tuning Needle 2 with LoRA requires understanding three core flags registered in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) at lines 127–128:

| Flag | Default | Purpose |
|------|---------|---------|
| `--lora-rank` | `16` | Rank `r` of the LoRA adapter matrices—determines the bottleneck dimension |
| `--lora-alpha` | `32.0` | Scaling factor `α`; effective merge scale is `α / r` |
| `--lora` | `None` | Path to an existing adapter file to merge before export |

The rank directly controls parameter count: lower ranks train faster with fewer parameters, while higher ranks capture more expressive capacity. The alpha scaling factor determines how strongly the adapter influences the base model during inference.

## How LoRA Fine-Tuning Works in Needle 2

The implementation in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) orchestrates a four-stage pipeline:

### 1. Target Selection (`lora_target_paths`)

At lines 54–63, Needle 2 automatically identifies which weight tensors to adapt. Only **linear kernels in attention layers** with non-trivial magnitude are selected, filtering out irrelevant parameters and ensuring efficient adaptation.

### 2. Adapter Initialization (`init_lora`)

Lines 67–81 create low-rank matrices for each target. For a kernel with input dimension `in_dim` and output dimension `out_dim`, Needle 2 initializes:

- Matrix **A**: shape `(..., in_dim, rank)` — the down-projection
- Matrix **B**: shape `(..., rank, out_dim)` — the up-projection

The rank value comes directly from your `--lora-rank` argument.

### 3. Merge Computation (`merge_lora`)

During the forward pass (lines 85–91), the adapter contribution is computed as:

```

scaled_contribution = (α / r) × (B @ A)

```

This low-rank update is added to the original frozen weights, enabling efficient training with full expressiveness at inference.

### 4. Optimizer Configuration

Only LoRA parameters enter the optimizer—base model weights remain frozen. This makes fine-tuning feasible on consumer hardware even for large models.

## Complete CLI Examples for Needle 2 LoRA Fine-Tuning

### Basic Fine-Tuning with Custom Rank

```bash
needle finetune \
  data/training.jsonl \
  --checkpoint meta/needle-2-base.pkl \
  --epochs 5 \
  --batch-size 32 \
  --lr 5e-5 \
  --lora-rank 8 \
  --lora-alpha 16 \
  --max-len 1024 \
  --out adapters/needle2_lora8.pkl

```

This configuration uses **rank 8**—half the default—for faster training with fewer parameters. Set `--lora-alpha 16` so the effective scaling equals `16 / 8 = 2.0`.

### Higher-Rank Fine-Tuning for Complex Tasks

```bash
needle finetune \
  data/training.jsonl \
  --checkpoint meta/needle-2-base.pkl \
  --epochs 10 \
  --batch-size 16 \
  --lr 1e-4 \
  --lora-rank 32 \
  --lora-alpha 64 \
  --max-len 2048 \
  --out adapters/needle2_lora32.pkl

```

Use `--lora-rank 32` with proportional `--lora-alpha 64` (maintaining `α/r = 2.0`) when the target task requires more representational capacity.

### Merging and Exporting a LoRA Adapter

After training, merge your adapter into the base model for deployment:

```bash
needle build meta/needle-2-base.pkl \
  --lora adapters/needle2_lora8.pkl \
  --out exported/needle2_lora8.cact

```

The `--lora` flag triggers adapter loading and permanent merging before export.

## Inspecting LoRA Adapter Files Programmatically

Needle 2 stores adapters as pickled dictionaries. Verify your training results:

```python
import pickle

with open('adapters/needle2_lora8.pkl', 'rb') as f:
    adapter = pickle.load(f)

print(f"LoRA rank: {adapter['rank']}")
print(f"Alpha value: {adapter.get('alpha', 'not stored')}")
print(f"Number of adapted weight groups: {len(adapter['lora'])}")

# Inspect shape of first adapter

first_key = list(adapter['lora'].keys())[0]
first_adapter = adapter['lora'][first_key]
print(f"First adapter keys: {list(first_adapter.keys())}")

```

## Key Source Files for LoRA Implementation

| File | Lines | Responsibility |
|------|-------|--------------|
| [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) | 127–128 | CLI argument parsing for `--lora-rank`, `--lora-alpha`, `--lora` |
| [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) | 54–63 | `lora_target_paths()` — attention layer target selection |
| [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) | 67–81 | `init_lora()` — adapter matrix initialization |
| [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) | 85–91 | `merge_lora()` — scaled forward pass computation |
| [`tests/test_finetune.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_finetune.py) | — | Integration tests for end-to-end LoRA training |
| [`tests/test_lora.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_lora.py) | — | Unit tests for target selection, init, and merge correctness |

## Choosing the Right `--lora-rank` Value

| Rank | Parameters | Best For |
|------|-----------|----------|
| 4–8 | Minimal | Quick experiments, simple style transfer, limited GPU memory |
| 16 (default) | Balanced | General-purpose fine-tuning, most production use cases |
| 32–64 | Higher | Complex reasoning tasks, substantial domain shifts, when overfitting is not a concern |

The default `--lora-rank 16` with `--lora-alpha 32` provides a 2.0 scaling factor that works well across diverse tasks. Adjust proportionally: if you halve the rank, consider halving alpha to maintain similar effective magnitude.

## Summary

- **Register flags** in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) — `--lora-rank`, `--lora-alpha`, and `--lora` control Needle 2's LoRA behavior
- **Initialize adapters** in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) with rank-dimensional matrices A and B
- **Merge during forward pass** using scale `α / r` computed from your CLI arguments
- **Optimize only adapters** — base model weights stay frozen for memory efficiency
- **Export merged models** with `needle build --lora <path>` for deployment without runtime adapter overhead

## Frequently Asked Questions

### What is the relationship between `--lora-rank` and `--lora-alpha` in Needle 2?

The effective scaling applied during merging equals `alpha / rank`. Needle 2 defaults to `rank=16` and `alpha=32`, giving a 2.0 scaling factor. This ratio determines how strongly the adapter influences predictions—lower ratios reduce adapter impact, higher ratios amplify it.

### Can I resume training or merge multiple LoRA adapters?

Needle 2 supports loading existing adapters via the `--lora` flag during `needle build`, which merges them permanently into the base model. For iterative training, load a previously saved adapter checkpoint and continue fine-tuning—though the current CLI exposes this most cleanly through the build-and-restart workflow.

### Why does Needle 2 only target attention layers for LoRA?

The `lora_target_paths` function in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) filters for attention linear kernels because empirical research and internal testing show these layers capture the most transferable, task-specific information. This selective targeting maximizes parameter efficiency—fewer adapted weights mean faster training and reduced overfitting risk.

### How do I verify my LoRA adapter trained correctly?

Check the saved pickle file as shown in the programmatic inspection example above. Confirm `adapter['rank']` matches your `--lora-rank` argument and that `len(adapter['lora'])` corresponds to the expected number of attention layers. The [`tests/test_lora.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_lora.py) file in the repository provides additional validation patterns for development workflows.