How to Synthesize Data for Needle 2 Fine-Tuning

TLDR: Needle 2 can be fine-tuned on custom tool-calling training data that you generate automatically using the needle generate-data CLI command or direct Python calls to the generate_dataset helper in needle.model.finetune.

Fine-tuning Needle 2 on specialized tool-calling tasks requires high-quality training examples. Instead of manual annotation, you can synthesize data for Needle 2 fine-tuning programmatically using the built-in generation workflow provided by the cactus-compute/needle repository. The toolchain interfaces with the OpenRouter API to produce diverse, realistic examples based on your JSON tool schemas.

Preparing Tool Schemas for Data Generation

Before invoking the synthesiser, you must define your tool specifications in a JSON file. This schema describes the function name, description, and argument structure that the model will learn to invoke.

JSON Schema Structure

The tool schema follows standard function-calling conventions. Each tool object includes the function name, a natural language description, and a parameters object defining required and optional arguments. The synthesiser uses these definitions to generate contextually appropriate queries and corresponding tool calls.

Running the Data Synthesiser

The primary entry point for data generation is the generate_main function in needle/model/finetune.py. You can access this functionality through the CLI for quick tasks or programmatically for custom integration.

CLI Data Generation

The generate-data sub-command defined in needle/cli.py provides the simplest interface. It accepts your tool schema file, target sample count, and OpenRouter model identifier.

Generate 200 new examples for your tools:

needle generate-data \
  --tools schemas.json \
  --num-samples 200 \
  --batch-size 25 \
  --workers 16 \
  --model deepseek/deepseek-v4-flash \
  --output synthetic_data.jsonl

Python API for Custom Pipelines

For integration into existing workflows, import the generation helpers directly from the finetune module. The generate_dataset function parallelises requests, deduplicates results, and manages the target sample count.

from needle.model.finetune import generate_dataset
import json

# Load your tool schemas (list of dicts)

with open("schemas.json") as f:
    tools = json.load(f)

# Produce 300 examples using the default OpenRouter model

samples = generate_dataset(
    tools,
    num_samples=300,
    model="deepseek/deepseek-v4-flash",
    batch_size=30,
    workers=8,
)

# Write to JSONL

with open("synthetic.jsonl", "w") as out:
    for ex in samples:
        out.write(json.dumps(ex) + "\n")

Augmenting Existing Datasets

If you already have a JSONL dataset, use the augment_jsonl function or the --augment CLI flag to expand it with new synthetic examples while preserving the original entries. The function extracts existing tool schemas from the file, generates additional samples, and writes a new augmented file.

Augment an existing dataset with 500 new examples:

needle generate-data \
  --augment my_data.jsonl \
  --num-samples 500 \
  --output my_data_augmented.jsonl

How the Synthesis Works Internally

Under the hood, the synthesiser relies on two core helpers in needle/model/finetune.py. The generate_examples function sends a single request to the OpenRouter API using the _GEN_SYSTEM system message and _GEN_TEMPLATE prompt template (defined at lines 37-54). The model is instructed to produce realistic user queries together with the exact tool calls that satisfy them.

The generate_dataset helper orchestrates this process by:

• Parallelising multiple generate_examples calls across worker threads
• Deduplicating results to ensure diversity
• Respecting the target num_samples count

Each generated JSON object contains:

• query – The natural-language request or passage
• reasoning – A short rationale linking the query to specific tool arguments
• answers – A list of precise tool calls with arguments matching the provided schemas

Summary

• Prepare tool schemas as JSON files describing your function signatures before generation
• Use needle generate-data for CLI-based synthesis or import generate_dataset from needle.model.finetune for Python scripts
• Leverage augment_jsonl or the --augment flag to expand existing datasets without losing original examples
• Generation uses OpenRouter API calls templated by _GEN_TEMPLATE and _GEN_SYSTEM in needle/model/finetune.py
• Output format includes query, reasoning, and answers fields for each synthetic example

Frequently Asked Questions

What API does Needle 2 use for data synthesis?

The synthesiser contacts the OpenRouter API to generate synthetic examples. You specify the model identifier (e.g., deepseek/deepseek-v4-flash) via the --model CLI flag or model parameter in the Python API.

Can I augment existing training data instead of generating from scratch?

Yes. Use the --augment flag followed by the path to your existing JSONL file, or call the augment_jsonl function directly in Python. This extracts the tool schemas from your current data and generates additional examples while preserving the original entries.

What fields are included in the generated synthetic examples?

Each generated example contains three fields: query (the natural language request), reasoning (the rationale connecting the query to tool arguments), and answers (a list of precise tool calls matching your schemas).

How does the synthesiser handle large-scale generation?

The generate_dataset helper parallelises requests across multiple workers (controlled by the --workers or workers parameter) and processes them in batches (controlled by --batch-size). It automatically deduplicates results to maintain dataset diversity while hitting the target sample count.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}