# Needle Finetune Command-Line Options: Complete Guide to LoRA Training

> Master needle finetune for LoRA training. Explore 15+ command-line options for model selection, hyperparameters, LoRA architecture, data, and quantization. Optimize your training today.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-06

---

**`needle finetune` trains LoRA adapters on JSONL datasets with 15+ configurable options covering model selection, training hyperparameters, LoRA architecture, data generation, and quantization.**

This guide walks through every command-line option for the `needle finetune` sub-command in the [cactus-compute/needle](https://github.com/cactus-compute/needle) repository. Whether you're fine-tuning a local checkpoint or generating synthetic training data, these options control the full pipeline defined in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) and executed by [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py).

## Required and Positional Arguments

### `jsonl_path`

The only required argument is the path to your training data.

```bash
needle finetune my_data.jsonl

```

This positional argument must point to a JSONL file containing training examples. Each line should be a JSON object with fields matching your model's expected input format.

## Core Training Hyperparameters

These options control the optimization loop in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py):

| Option | Default | Purpose |
|--------|---------|---------|
| `--epochs` | `3` | Number of complete passes through the dataset |
| `--batch-size` | `16` | Samples per gradient update |
| `--lr` | `1e-4` | Base learning rate for the AdamW optimizer |

Adjust these based on dataset size and convergence behavior:

```bash

# Longer training with larger batches for a big dataset

needle finetune large_dataset.jsonl --epochs 10 --batch-size 64 --lr 3e-5

```

## LoRA Adapter Configuration

LoRA (Low-Rank Adaptation) reduces trainable parameters by injecting rank-decomposed matrices. Two key options control its behavior:

- **`--lora-rank`** (default: `16`) — Dimension of the low-rank matrices. Higher values increase capacity but memory. Typical range: 4–64.
- **`--lora-alpha`** (default: `32.0`) — Scaling factor. **Effective learning rate scales as `alpha / rank`**, so `alpha=32, rank=16` gives a 2× multiplier on LoRA updates.

```bash

# Higher-capacity adapter with proportional scaling

needle finetune data.jsonl --lora-rank 32 --lora-alpha 64.0

```

## Model Loading and Checkpointing

### `--checkpoint`

Specifies a local base model. If omitted, the checkpoint downloads automatically from Hugging Face:

```bash

# Use local checkpoint

needle finetune data.jsonl --checkpoint ./my_base.pkl

# Auto-download from Hugging Face Hub

needle finetune data.jsonl

```

### `--checkpoint-dir`

Cache location for downloaded models (default: `checkpoints`).

### `--out`

Destination path for the trained LoRA adapter. If unset, the adapter saves with a generated name:

```bash
needle finetune data.jsonl --out ./adapters/my_task_lora.pkl

```

## Data Processing Options

| Option | Default | Description |
|--------|---------|-------------|
| `--max-len` | `1024` | Maximum token sequence length (truncation/padding applied) |
| `--val-split` | `0.1` | Fraction reserved for validation; set to `0` to disable |

Longer sequences increase memory usage quadratically with attention. Reduce `--max-len` if encountering OOM errors:

```bash

# Short sequences, no validation

needle finetune data.jsonl --max-len 512 --val-split 0

```

## Synthetic Data Generation

The `--generate` option augments training data via OpenRouter API calls before fine-tuning begins:

| Option | Default | Purpose |
|--------|---------|---------|
| `--generate` | `0` | Number of synthetic examples to create (0 = disabled) |
| `--model` | `deepseek/deepseek-v4-flash` | OpenRouter model for generation |
| `--workers` | `8` | Concurrent API requests |

Requires `OPENROUTER_API_KEY` in environment:

```bash
export OPENROUTER_API_KEY="sk-or-..."

# Generate 500 examples then train

needle finetune seed_data.jsonl --generate 500 --model gpt-4o --workers 16

```

Generation happens in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) before the main training loop starts.

## Quantization-Aware Training (QAT)

The `--qat-bits` option controls numerical precision during training:

| Value | Behavior |
|-------|----------|
| `auto` | Match the checkpoint's export quantization |
| `none` | Full precision (FP32/FP16) training |
| `2` | Force 2-bit QAT |
| `4` | Force 4-bit QAT |

```bash

# Force full precision for maximum stability

needle finetune data.jsonl --qat-bits none

# Aggressive quantization for edge deployment

needle finetune data.jsonl --qat-bits 2

```

## Complete Example Configurations

### Local Development

```bash
needle finetune dev_set.jsonl \
  --epochs 1 \
  --batch-size 4 \
  --max-len 256 \
  --val-split 0 \
  --out quick_test.pkl

```

### Production Fine-Tuning

```bash
needle finetune production.jsonl \
  --checkpoint-dir /workspace/checkpoints \
  --epochs 5 \
  --batch-size 32 \
  --lr 5e-5 \
  --lora-rank 24 \
  --lora-alpha 48.0 \
  --max-len 2048 \
  --qat-bits 4 \
  --out ./models/task_adapter.pkl

```

### Synthetic Augmentation Pipeline

```bash
export OPENROUTER_API_KEY="sk-or-..."

needle finetune sparse_seed.jsonl \
  --generate 1000 \
  --model anthropic/claude-3-sonnet-20240229 \
  --workers 20 \
  --epochs 3 \
  --batch-size 16

```

## Option Reference Table

| Option | Type | Default | Where Used |
|--------|------|---------|------------|
| `jsonl_path` | positional `str` | — | [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) |
| `--checkpoint` | `str` | `None` | Checkpoint loading logic |
| `--epochs` | `int` | `3` | Training loop iterations |
| `--batch-size` | `int` | `16` | DataLoader configuration |
| `--lr` | `float` | `1e-4` | Optimizer init |
| `--lora-rank` | `int` | `16` | LoRA matrix dimensions |
| `--lora-alpha` | `float` | `32.0` | LoRA scaling factor |
| `--max-len` | `int` | `1024` | Tokenization padding |
| `--val-split` | `float` | `0.1` | Train/validation split |
| `--generate` | `int` | `0` | Synthetic data creation |
| `--model` | `str` | `deepseek/deepseek-v4-flash` | OpenRouter model selection |
| `--workers` | `int` | `8` | Concurrent API requests |
| `--checkpoint-dir` | `str` | `checkpoints` | HuggingFace cache path |
| `--out` | `str` | `None` | Adapter save location |
| `--qat-bits` | `auto\|none\|2\|4` | `auto` | Quantization config |

Source: [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) lines 31–52, [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) lines 94–107.

## Summary

- **Required**: Only `jsonl_path` is mandatory; all other options have sensible defaults
- **LoRA tuning**: Adjust `--lora-rank` and `--lora-alpha` together, maintaining your desired `alpha/rank` ratio
- **Memory management**: Reduce `--batch-size` or `--max-len` if GPU memory is constrained
- **Data augmentation**: Combine `--generate` with `--workers` for scalable synthetic training sets
- **Precision control**: Use `--qat-bits auto` for most cases, `none` for debugging, `2`/`4` for deployment

## Frequently Asked Questions

### What happens if I don't specify a checkpoint?

The pipeline automatically downloads the default model from Hugging Face Hub into `--checkpoint-dir`. No manual download is required.

### How do I choose LoRA rank and alpha values?

**Higher rank** (32–64) captures more complex adaptations but increases parameters and memory. **Alpha** should typically be 2× the rank for balanced scaling. For small datasets or simple tasks, rank 8–16 often suffices.

### Can I disable validation entirely?

Yes. Set `--val-split 0` to use 100% of data for training. This speeds up epochs but removes early stopping and loss monitoring capabilities.

### Why does synthetic generation require OpenRouter?

The `--generate` option calls proprietary models through OpenRouter's unified API. The `OPENROUTER_API_KEY` environment variable authenticates these requests. Generation runs before any local training begins, augmenting your JSONL with model-written examples.