Needle Finetune Command-Line Options: Complete Guide to LoRA Training

needle finetune trains LoRA adapters on JSONL datasets with 15+ configurable options covering model selection, training hyperparameters, LoRA architecture, data generation, and quantization.

This guide walks through every command-line option for the needle finetune sub-command in the cactus-compute/needle repository. Whether you're fine-tuning a local checkpoint or generating synthetic training data, these options control the full pipeline defined in needle/cli.py and executed by needle/model/finetune.py.

Required and Positional Arguments

jsonl_path

The only required argument is the path to your training data.

needle finetune my_data.jsonl

This positional argument must point to a JSONL file containing training examples. Each line should be a JSON object with fields matching your model's expected input format.

Core Training Hyperparameters

These options control the optimization loop in needle/model/finetune.py:

Option	Default	Purpose
--epochs	3	Number of complete passes through the dataset
--batch-size	16	Samples per gradient update
--lr	1e-4	Base learning rate for the AdamW optimizer

Adjust these based on dataset size and convergence behavior:

# Longer training with larger batches for a big dataset

needle finetune large_dataset.jsonl --epochs 10 --batch-size 64 --lr 3e-5

LoRA Adapter Configuration

LoRA (Low-Rank Adaptation) reduces trainable parameters by injecting rank-decomposed matrices. Two key options control its behavior:

• --lora-rank (default: 16) — Dimension of the low-rank matrices. Higher values increase capacity but memory. Typical range: 4–64.
• --lora-alpha (default: 32.0) — Scaling factor. Effective learning rate scales as alpha / rank, so alpha=32, rank=16 gives a 2× multiplier on LoRA updates.

# Higher-capacity adapter with proportional scaling

needle finetune data.jsonl --lora-rank 32 --lora-alpha 64.0

Model Loading and Checkpointing

--checkpoint

Specifies a local base model. If omitted, the checkpoint downloads automatically from Hugging Face:

# Use local checkpoint

needle finetune data.jsonl --checkpoint ./my_base.pkl

# Auto-download from Hugging Face Hub

needle finetune data.jsonl

--checkpoint-dir

Cache location for downloaded models (default: checkpoints).

--out

Destination path for the trained LoRA adapter. If unset, the adapter saves with a generated name:

needle finetune data.jsonl --out ./adapters/my_task_lora.pkl

Data Processing Options

Option	Default	Description
--max-len	1024	Maximum token sequence length (truncation/padding applied)
--val-split	0.1	Fraction reserved for validation; set to 0 to disable

Longer sequences increase memory usage quadratically with attention. Reduce --max-len if encountering OOM errors:

# Short sequences, no validation

needle finetune data.jsonl --max-len 512 --val-split 0

Synthetic Data Generation

The --generate option augments training data via OpenRouter API calls before fine-tuning begins:

Option	Default	Purpose
--generate	0	Number of synthetic examples to create (0 = disabled)
--model	deepseek/deepseek-v4-flash	OpenRouter model for generation
--workers	8	Concurrent API requests

Requires OPENROUTER_API_KEY in environment:

export OPENROUTER_API_KEY="sk-or-..."

# Generate 500 examples then train

needle finetune seed_data.jsonl --generate 500 --model gpt-4o --workers 16

Generation happens in needle/model/finetune.py before the main training loop starts.

Quantization-Aware Training (QAT)

The --qat-bits option controls numerical precision during training:

Value	Behavior
auto	Match the checkpoint's export quantization
none	Full precision (FP32/FP16) training
2	Force 2-bit QAT
4	Force 4-bit QAT

# Force full precision for maximum stability

needle finetune data.jsonl --qat-bits none

# Aggressive quantization for edge deployment

needle finetune data.jsonl --qat-bits 2

Complete Example Configurations

Local Development

needle finetune dev_set.jsonl \
  --epochs 1 \
  --batch-size 4 \
  --max-len 256 \
  --val-split 0 \
  --out quick_test.pkl

Production Fine-Tuning

needle finetune production.jsonl \
  --checkpoint-dir /workspace/checkpoints \
  --epochs 5 \
  --batch-size 32 \
  --lr 5e-5 \
  --lora-rank 24 \
  --lora-alpha 48.0 \
  --max-len 2048 \
  --qat-bits 4 \
  --out ./models/task_adapter.pkl

Synthetic Augmentation Pipeline

export OPENROUTER_API_KEY="sk-or-..."

needle finetune sparse_seed.jsonl \
  --generate 1000 \
  --model anthropic/claude-3-sonnet-20240229 \
  --workers 20 \
  --epochs 3 \
  --batch-size 16

Option Reference Table

Option	Type	Default	Where Used
jsonl_path	positional str	—	needle/cli.py, needle/model/finetune.py
--checkpoint	str	None	Checkpoint loading logic
--epochs	int	3	Training loop iterations
--batch-size	int	16	DataLoader configuration
--lr	float	1e-4	Optimizer init
--lora-rank	int	16	LoRA matrix dimensions
--lora-alpha	float	32.0	LoRA scaling factor
--max-len	int	1024	Tokenization padding
--val-split	float	0.1	Train/validation split
--generate	int	0	Synthetic data creation
--model	str	deepseek/deepseek-v4-flash	OpenRouter model selection
--workers	int	8	Concurrent API requests
--checkpoint-dir	str	checkpoints	HuggingFace cache path
--out	str	None	Adapter save location
--qat-bits	auto|none|2|4	auto	Quantization config

Source: needle/cli.py lines 31–52, needle/model/finetune.py lines 94–107.

Summary

• Required: Only jsonl_path is mandatory; all other options have sensible defaults
• LoRA tuning: Adjust --lora-rank and --lora-alpha together, maintaining your desired alpha/rank ratio
• Memory management: Reduce --batch-size or --max-len if GPU memory is constrained
• Data augmentation: Combine --generate with --workers for scalable synthetic training sets
• Precision control: Use --qat-bits auto for most cases, none for debugging, 2/4 for deployment

Frequently Asked Questions

What happens if I don't specify a checkpoint?

The pipeline automatically downloads the default model from Hugging Face Hub into --checkpoint-dir. No manual download is required.

How do I choose LoRA rank and alpha values?

Higher rank (32–64) captures more complex adaptations but increases parameters and memory. Alpha should typically be 2× the rank for balanced scaling. For small datasets or simple tasks, rank 8–16 often suffices.

Can I disable validation entirely?

Yes. Set --val-split 0 to use 100% of data for training. This speeds up epochs but removes early stopping and loss monitoring capabilities.

Why does synthetic generation require OpenRouter?

The --generate option calls proprietary models through OpenRouter's unified API. The OPENROUTER_API_KEY environment variable authenticates these requests. Generation runs before any local training begins, augmenting your JSONL with model-written examples.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}