Data Format Required for Fine-Tuning Needle 2: JSONL Schema and Constraints

Needle 2 requires a JSONL file where each line contains a JSON object with query, tools, and answers fields, following strict constraints that argument values must appear verbatim in the source text.

Needle 2 leverages LoRA adapters to learn tool-calling JSON generation from natural language prompts. The data format required for fine-tuning Needle 2 is a strict JSONL schema implemented in the cactus-compute/needle repository, with validation logic located in needle/model/finetune.py.

JSONL Schema Structure

Each line in the training file must contain a single JSON object with specific required and optional fields.

Required Fields

Every training example must include these three fields:

• query: The user's raw request or the source passage from which information is extracted.
• tools: A list of tool definitions that apply to the example, using the same schema as the inference engine. This field is automatically populated by the data generator.
• answers: An array of exact tool calls the model should produce. Each entry contains name (the tool identifier) and arguments (a dictionary of extracted values).

Optional Fields

These fields enhance training quality but are not strictly enforced by the parser:

• reasoning: A single short line demonstrating how each argument was derived from the query. According to _parse_array in needle/model/finetune.py, this field provides grounding during training and is strongly recommended though not mandatory.
• system: A system prompt prepended to the example, matching the Needle(system=...) parameter used at inference time.

Critical Data Constraints

The training pipeline enforces four key constraints to ensure model quality:

Argument values must appear verbatim in the query. When extracting information, the model must transcribe text exactly as it appears in the source. Optional fields lacking evidence must be omitted entirely—never use placeholders or empty strings.

Include "off-topic" examples. Training data must contain examples where answers is an empty list. This prevents the model from over-triggering tools on irrelevant inputs.

Provide ambiguous queries for similar tools. When multiple tools share similar capabilities, include deliberately ambiguous queries that resolve to the specific correct tool, teaching the model to distinguish between them.

Respect the token limit. Every example must fit within the --max-len token window, which defaults to 1024 tokens. The parser silently truncates longer examples, potentially corrupting training data.

Validation and Parsing Implementation

The _parse_array function in needle/model/finetune.py handles data validation during training initialization. Lines 69-78 explicitly check for the presence of query and answers fields, discarding any rows that lack these mandatory elements. This validation ensures schema compliance before data enters the LoRA training loop.

Practical Examples

Minimal JSONL Entry

A valid training example fits on a single line:

{"query":"Bantilan, N. (2018). Themis. Journal of Technology in Human Services, 36(1).","tools":[{"name":"extract_citation_data","parameters":{"type":"object","properties":{"authors":{"type":"string"},"title":{"type":"string"},"publisher":{"type":"string"}},"required":["authors","title"]}}],"answers":[{"name":"extract_citation_data","arguments":{"authors":"Bantilan, N.","title":"Themis","publisher":"Journal of Technology in Human Services, 36(1)."}}],"reasoning":"authors precede the year; title follows the year; publisher is the journal segment"}

Generating Synthetic Data

Create training data from an existing tool schema:

# Generate 1000 synthetic examples from a tool schema file (tools.json)

needle generate-data --augment data.jsonl --num-samples 1000

Running Fine-Tuning

Train and export the adapter:

# Train LoRA adapter on the JSONL file

needle finetune data.jsonl --epochs 10 --out adapter.pkl

# Merge the adapter into the base model and export a tuned package

needle build checkpoints/needle2.pkl --lora adapter.pkl --out tuned.cact

Summary

• Needle 2 requires JSONL format with one JSON object per line.
• Mandatory fields are query, tools, and answers; optional fields include reasoning and system.
• Argument values must appear verbatim in the query text.
• Include off-topic examples with empty answers to reduce false positives.
• The _parse_array function in needle/model/finetune.py validates data during training initialization.
• Examples exceeding 1024 tokens (configurable via --max-len) are silently truncated.

Frequently Asked Questions

What file format does Needle 2 require for fine-tuning?

Needle 2 requires a JSONL (JSON Lines) file where each line contains a single JSON object. The parser in needle/model/finetune.py processes this file line-by-line, rejecting any rows missing the required query or answers fields at lines 69-78.

Is the reasoning field mandatory in Needle 2 training data?

No, the reasoning field is optional but strongly recommended. This field provides a single line explaining how arguments were derived from the query, which improves model grounding during LoRA training. The validation logic in _parse_array does not enforce its presence, though doc/finetuning.md advises its inclusion.

How does Needle 2 handle arguments that don't appear in the query?

Arguments without evidence in the query must be omitted entirely from the arguments dictionary. The schema strictly prohibits placeholder values or empty strings for optional fields. This constraint ensures the model learns to extract only verified information from source text.

What happens if a training example exceeds the maximum token length?

Examples exceeding the --max-len threshold (default 1024 tokens) are silently truncated during training. This truncation occurs before data enters the training loop in needle/model/finetune.py, potentially cutting off critical information. Always verify example lengths stay within the token limit to prevent data corruption.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}