How to Fine-Tune Needle 2 with LoRA Using the CLI and Key Parameters Like `--lora-rank`

Use needle finetune with --lora-rank to set the low-rank dimension (default: 16), --lora-alpha to control scaling, and --lora to load existing adapters for merging—enabling parameter-efficient fine-tuning without modifying base model weights.

The Needle 2 framework from cactus-compute/needle provides a streamlined command-line interface for LoRA (Low-Rank Adaptation) fine-tuning. This approach dramatically reduces memory and compute requirements by training only small adapter matrices while keeping the base model frozen. The CLI exposes precise control over rank, scaling, and target layer selection through well-documented flags.

Understanding LoRA Parameters in Needle 2

Fine-tuning Needle 2 with LoRA requires understanding three core flags registered in needle/cli.py at lines 127–128:

Flag	Default	Purpose
--lora-rank	16	Rank r of the LoRA adapter matrices—determines the bottleneck dimension
--lora-alpha	32.0	Scaling factor α; effective merge scale is α / r
--lora	None	Path to an existing adapter file to merge before export

The rank directly controls parameter count: lower ranks train faster with fewer parameters, while higher ranks capture more expressive capacity. The alpha scaling factor determines how strongly the adapter influences the base model during inference.

How LoRA Fine-Tuning Works in Needle 2

The implementation in needle/model/finetune.py orchestrates a four-stage pipeline:

1. Target Selection (lora_target_paths)

At lines 54–63, Needle 2 automatically identifies which weight tensors to adapt. Only linear kernels in attention layers with non-trivial magnitude are selected, filtering out irrelevant parameters and ensuring efficient adaptation.

2. Adapter Initialization (init_lora)

Lines 67–81 create low-rank matrices for each target. For a kernel with input dimension in_dim and output dimension out_dim, Needle 2 initializes:

• Matrix A: shape (..., in_dim, rank) — the down-projection
• Matrix B: shape (..., rank, out_dim) — the up-projection

The rank value comes directly from your --lora-rank argument.

3. Merge Computation (merge_lora)

During the forward pass (lines 85–91), the adapter contribution is computed as:

scaled_contribution = (α / r) × (B @ A)

This low-rank update is added to the original frozen weights, enabling efficient training with full expressiveness at inference.

4. Optimizer Configuration

Only LoRA parameters enter the optimizer—base model weights remain frozen. This makes fine-tuning feasible on consumer hardware even for large models.

Complete CLI Examples for Needle 2 LoRA Fine-Tuning

Basic Fine-Tuning with Custom Rank

needle finetune \
  data/training.jsonl \
  --checkpoint meta/needle-2-base.pkl \
  --epochs 5 \
  --batch-size 32 \
  --lr 5e-5 \
  --lora-rank 8 \
  --lora-alpha 16 \
  --max-len 1024 \
  --out adapters/needle2_lora8.pkl

This configuration uses rank 8—half the default—for faster training with fewer parameters. Set --lora-alpha 16 so the effective scaling equals 16 / 8 = 2.0.

Higher-Rank Fine-Tuning for Complex Tasks

needle finetune \
  data/training.jsonl \
  --checkpoint meta/needle-2-base.pkl \
  --epochs 10 \
  --batch-size 16 \
  --lr 1e-4 \
  --lora-rank 32 \
  --lora-alpha 64 \
  --max-len 2048 \
  --out adapters/needle2_lora32.pkl

Use --lora-rank 32 with proportional --lora-alpha 64 (maintaining α/r = 2.0) when the target task requires more representational capacity.

Merging and Exporting a LoRA Adapter

After training, merge your adapter into the base model for deployment:

needle build meta/needle-2-base.pkl \
  --lora adapters/needle2_lora8.pkl \
  --out exported/needle2_lora8.cact

The --lora flag triggers adapter loading and permanent merging before export.

Inspecting LoRA Adapter Files Programmatically

Needle 2 stores adapters as pickled dictionaries. Verify your training results:

import pickle

with open('adapters/needle2_lora8.pkl', 'rb') as f:
    adapter = pickle.load(f)

print(f"LoRA rank: {adapter['rank']}")
print(f"Alpha value: {adapter.get('alpha', 'not stored')}")
print(f"Number of adapted weight groups: {len(adapter['lora'])}")

# Inspect shape of first adapter

first_key = list(adapter['lora'].keys())[0]
first_adapter = adapter['lora'][first_key]
print(f"First adapter keys: {list(first_adapter.keys())}")

Key Source Files for LoRA Implementation

File	Lines	Responsibility
needle/cli.py	127–128	CLI argument parsing for --lora-rank, --lora-alpha, --lora
needle/model/finetune.py	54–63	lora_target_paths() — attention layer target selection
needle/model/finetune.py	67–81	init_lora() — adapter matrix initialization
needle/model/finetune.py	85–91	merge_lora() — scaled forward pass computation
tests/test_finetune.py	—	Integration tests for end-to-end LoRA training
tests/test_lora.py	—	Unit tests for target selection, init, and merge correctness

Choosing the Right --lora-rank Value

Rank	Parameters	Best For
4–8	Minimal	Quick experiments, simple style transfer, limited GPU memory
16 (default)	Balanced	General-purpose fine-tuning, most production use cases
32–64	Higher	Complex reasoning tasks, substantial domain shifts, when overfitting is not a concern

The default --lora-rank 16 with --lora-alpha 32 provides a 2.0 scaling factor that works well across diverse tasks. Adjust proportionally: if you halve the rank, consider halving alpha to maintain similar effective magnitude.

Summary

• Register flags in needle/cli.py — --lora-rank, --lora-alpha, and --lora control Needle 2's LoRA behavior
• Initialize adapters in needle/model/finetune.py with rank-dimensional matrices A and B
• Merge during forward pass using scale α / r computed from your CLI arguments
• Optimize only adapters — base model weights stay frozen for memory efficiency
• Export merged models with needle build --lora <path> for deployment without runtime adapter overhead

Frequently Asked Questions

What is the relationship between --lora-rank and --lora-alpha in Needle 2?

The effective scaling applied during merging equals alpha / rank. Needle 2 defaults to rank=16 and alpha=32, giving a 2.0 scaling factor. This ratio determines how strongly the adapter influences predictions—lower ratios reduce adapter impact, higher ratios amplify it.

Can I resume training or merge multiple LoRA adapters?

Needle 2 supports loading existing adapters via the --lora flag during needle build, which merges them permanently into the base model. For iterative training, load a previously saved adapter checkpoint and continue fine-tuning—though the current CLI exposes this most cleanly through the build-and-restart workflow.

Why does Needle 2 only target attention layers for LoRA?

The lora_target_paths function in needle/model/finetune.py filters for attention linear kernels because empirical research and internal testing show these layers capture the most transferable, task-specific information. This selective targeting maximizes parameter efficiency—fewer adapted weights mean faster training and reduced overfitting risk.

How do I verify my LoRA adapter trained correctly?

Check the saved pickle file as shown in the programmatic inspection example above. Confirm adapter['rank'] matches your --lora-rank argument and that len(adapter['lora']) corresponds to the expected number of attention layers. The tests/test_lora.py file in the repository provides additional validation patterns for development workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}