How to Prepare Data for LoRA Fine-Tuning in Needle: A Complete Guide

Needle trains LoRA adapters on JSONL files where each line contains a user query, tool schemas, and the exact function calls the model should generate, with all arguments strictly grounded in the query text.

Needle is an open-source tool-calling framework that fine-tunes frozen base models by injecting small, trainable LoRA (Low-Rank Adaptation) adapters. To prepare data for LoRA fine-tuning in Needle, you must create structured JSONL examples that teach the model when and how to invoke tools based on natural language inputs, as implemented in the cactus-compute/needle repository.

JSONL Schema for Training Examples

Each line in your training file must be a single JSON object describing one interaction turn. According to doc/finetuning.md and the parser in needle/model/finetune.py, the object requires three core fields and supports two optional modifiers.

Required Fields

• query: The natural language user request or passage to extract from.
• tools: An array of tool schema objects (identical to what you pass to Needle(tools=…)).
• answers: An array of function call objects containing the exact name and arguments the model must emit. Arguments must be substrings drawn directly from the query.

Optional Fields

• reasoning: A short explanation showing how each argument maps to a span in the query. This teaches the model to ground its outputs and reduces hallucinations.
• system: If present, the example is treated as a system turn (equivalent to Needle(system=…)).

Minimal Example

{"query":"Bantilan, N. (2018). Themis. Journal of Technology in Human Services, 36(1).",
 "tools":[{"name":"extract_citation_data","parameters":{"type":"object","properties":{"authors":{"type":"string"},"title":{"type":"string"},"publisher":{"type":"string"}},"required":["authors","title"]}}],
 "answers":[{"name":"extract_citation_data","arguments":{"authors":"Bantilan, N.","title":"Themis","publisher":"Journal of Technology in Human Services, 36(1)."}}],
 "reasoning":"authors precede the year; title follows the year; publisher is the journal segment"}

Critical Rules for Dataset Creation

The finetune.py module enforces several constraints during tokenization and training. Violating these rules produces ineffective adapters.

1. Grounded Arguments Only: Values in arguments must appear verbatim in the query. Omit optional fields when evidence is missing; never use placeholders like "N/A" or empty strings.
2. Include Negative Examples: Roughly one in eight examples should have "answers": [] to teach the model when no tool applies.
3. Ambiguity Resolution: When multiple tools are similar, include queries that could theoretically match several but resolve to one correct choice.
4. Token Budget Compliance: Keep examples within the max_len token budget (default 1024). Longer sequences are silently truncated, so shorter examples train faster and retain more signal.
5. Frozen Components: The confidence head and tokenizer are not modified by LoRA training. The head is disabled for tuned weights, and tokenization remains identical to the base model.

Step-by-Step Data Preparation Workflow

1. Seed Your Dataset

Create a hand-crafted JSONL file with 50–100 high-quality examples that cover your target domain and edge cases.

2. Augment with Synthetic Data

Use the built-in generator to expand your dataset. The generator queries DeepSeek-V4 (via OpenRouter) with a system prompt that enforces the rules above.

export OPENROUTER_API_KEY=your_key_here
needle generate-data --augment seed_data.jsonl --num-samples 1000

# Outputs seed_data.augmented.jsonl after deduplication

3. Train the LoRA Adapter

Run the fine-tuning CLI, which parses the JSONL, tokenizes prompts using needle/model/tokenizer.py, and performs gradient updates on adapter weights only (rank 16 by default).

needle finetune data.jsonl --epochs 10 --out adapter.pkl

4. Merge and Export

Combine the adapter with the base checkpoint into a portable .cact archive.

needle build checkpoints/needle2.pkl --lora adapter.pkl --out tuned.cact

Python API for Advanced Workflows

For programmatic pipelines, import the core functions from needle.model.finetune instead of using the CLI.

from needle.model.finetune import augment_jsonl, finetune_local
import argparse

# 1️⃣ Generate synthetic examples

aug_path = augment_jsonl("handwritten.jsonl", num_samples=2000)

# 2️⃣ Configure training arguments

parser = argparse.ArgumentParser()
parser.add_argument("--checkpoint", default="checkpoints/needle2.pkl")
parser.add_argument("--jsonl_path", default=aug_path)
parser.add_argument("--epochs", type=int, default=10)
parser.add_argument("--lora_rank", type=int, default=16)
parser.add_argument("--lora_alpha", type=float, default=32)
parser.add_argument("--batch_size", type=int, default=16)
parser.add_argument("--lr", type=float, default=0.0001)
parser.add_argument("--max_len", type=int, default=1024)
args = parser.parse_args([])

# 3️⃣ Execute local training

finetune_local(args)

Loading the Tuned Model

After building the tuned.cact archive, load it by pointing the weights parameter to the archive path. You must provide the same tool schema list used during training.

import needle

agent = needle.Needle(
    tools=[...],                # Identical schema list from training

    weights="tuned.cact"        # LoRA-enhanced checkpoint

)

Summary

• Needle requires JSONL files with query, tools, and answers fields to prepare data for LoRA fine-tuning.
• Arguments must be grounded in the query text; synthetic augmentation via needle generate-data speeds up dataset creation using DeepSeek-V4.
• Training occurs in needle/model/finetune.py with default rank 16 and alpha 32, leaving the tokenizer and confidence head frozen.
• Use needle finetune to create adapter.pkl, then needle build --lora to merge it into a deployable .cact file.
• Deploy by initializing Needle(weights="tuned.cact", tools=...) with the original tool definitions.

Frequently Asked Questions

What file format does Needle use for LoRA fine-tuning?

Needle uses JSONL (JSON Lines), where each line is a single JSON object representing one training example. This format is parsed by the finetune_local function in needle/model/finetune.py and supports streaming ingestion for large datasets.

How does the data generator work?

The needle generate-data CLI sends your seed examples to an LLM (default DeepSeek-V4 via OpenRouter) with a system prompt that enforces Needle’s schema and grounded-argument rules. The generator receives a JSON array of synthetic examples, deduplicates them, and appends them to your training file.

Why must arguments be drawn only from the query text?

This constraint teaches the model grounded extraction rather than hallucination. By requiring that every value in arguments appear verbatim in query, the LoRA adapter learns to identify and copy spans rather than generate plausible but incorrect text, which is critical for reliable tool calling.

What hyperparameters control the LoRA adapter training?

The rank (16 by default) and alpha (32 by default) control the adapter’s capacity and scaling. You can adjust these via --lora_rank and --lora_alpha in the CLI or the argparse namespace when calling finetune_local. Additional tunable parameters include --max_len (1024 tokens), --batch_size (16), and --lr (0.0001).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}