How to Synthesize Training Data with OpenRouter for Needle: Dataset Size and Generation Guide

You can synthesize training data for Needle by running needle generate-data with an OPENROUTER_API_KEY, providing a tool schema JSON file, and requesting hundreds to thousands of samples depending on whether you are training for tool selection (300–500 samples) or argument grounding (1,000–5,000 samples).

Needle is an open-source framework (cactus-compute/needle) for fine-tuning language models to reliably invoke tools. To teach the model which function to call and how to populate its parameters, you must furnish a JSONL dataset of user queries paired with correct tool invocations. The most efficient way to build this corpus is to synthesize training data with OpenRouter using Needle's built-in generator, which converts tool schemas into realistic conversational examples without manual labeling.

Configuring OpenRouter Authentication

Before invoking the generator, export your API credentials. The generate-data command expects an OPENROUTER_API_KEY environment variable, and optionally OPENROUTER_URL if you are routing through a custom gateway.

export OPENROUTER_API_KEY=sk-or-...

# Optional: export OPENROUTER_URL=https://custom.gateway/v1

Generating Synthetic Training Data with generate-data

The generate-data CLI command contacts the OpenRouter API (or any OpenAI-compatible endpoint), reads your tool definitions, and emits synthetic query-tool pairs. According to the README.md (lines 87-93), this is the primary mechanism for populating your training corpus.

Creating a Dataset from Scratch

Point --tools at a JSON schema describing your function catalogue, set --num-samples to your desired volume, and specify --output for the destination JSONL file.

needle generate-data \
  --tools my_tools.json \
  --num-samples 800 \
  --output training_data.jsonl

The generator explores linguistic variations around your tool definitions, producing diverse phrasings and realistic argument values.

Augmenting Existing Seed Data

If you already possess a hand-crafted dataset, expand it with the --augment flag. This appends new synthetic examples to your existing file rather than overwriting it.

needle generate-data \
  --augment existing_data.jsonl \
  --num-samples 500

Dataset Size Requirements for Needle Fine-Tuning

Not all training objectives require the same volume of data. As documented in doc/finetuning.md (lines 68-71), Needle distinguishes between tool selection (picking the correct function) and argument grounding (correctly filling parameters).

Tool Selection vs. Argument Grounding

• Tool selection improves rapidly with a few hundred high-quality examples. The model learns to map user intent to the appropriate function name.
• Argument grounding requires thousands of varied examples. The model must learn to extract specific values from the query text and map them to the correct JSON fields, often captured in the optional reasoning field.

Recommended Sample Counts by Catalog Size

Scale your dataset according to the complexity of your tool catalogue:

• Small catalogues (≤10 tools):

  • 300–500 synthetic samples for reliable tool selection
  • 1,000–2,000 additional samples for robust argument grounding

• Large catalogues (≥20 tools):

  • 1,000–2,000 samples for selection
  • 3,000–5,000 samples for grounding

For large catalogues, consider splitting the process into two inference passes: first train for full-catalogue selection, then train a separate model (or second phase) for single-tool grounding.

Data Format and Off-Topic Examples

Each line in the output JSONL contains query, tools, answers, and an optional reasoning field. The answers array holds the target tool calls.

Include a subset of off-topic examples where "answers": []. These teach the model to abstain from tool invocation when the user query is irrelevant to the available tools.

Example generated record:

{
  "query": "Dim the kitchen lights to 10 percent",
  "tools": [
    {
      "name": "set_lights",
      "description": "Set a room's light brightness.",
      "parameters": {
        "type": "object",
        "properties": {
          "room": {"type": "string"},
          "brightness": {"type": "integer"}
        },
        "required": ["room", "brightness"]
      }
    }
  ],
  "answers": [
    {
      "name": "set_lights",
      "arguments": {"room": "kitchen", "brightness": 10}
    }
  ],
  "reasoning": "'kitchen' → room; '10' → brightness"
}

Summary

• Set credentials: Export OPENROUTER_API_KEY before running Needle's generator.
• Generate data: Use needle generate-data --tools schema.json --num-samples N to create synthetic training pairs.
• Augment seeds: Append to existing datasets with the --augment flag.
• Size appropriately: Allocate 300–500 samples for tool selection and 1,000–5,000 for argument grounding, scaling with catalogue size.
• Include negatives: Add examples with empty answers arrays to teach abstention.

Frequently Asked Questions

Do I need to manually label data to train Needle?

No. While you can hand-craft examples, the generate-data command automates synthesis via OpenRouter, producing varied, realistic query-tool pairs from your JSON schema definitions. This eliminates the need for manual annotation while maintaining high data quality.

Can I use a different LLM provider instead of OpenRouter?

Yes. Export OPENROUTER_URL to point at any OpenAI-compatible API gateway. The generator uses standard chat completions, so compatible endpoints like OpenAI, Groq, or local vLLM instances work interchangeably with the same generate-data interface.

How does Needle use the reasoning field in training data?

The optional reasoning field documents how argument values map to spans in the user query. Including this textual explanation improves the model's argument grounding accuracy by providing explicit supervision for parameter extraction, showing the model exactly which words in the query correspond to each JSON value.

What happens if I provide too few samples for argument grounding?

With insufficient examples (fewer than 1,000 for small catalogues), the model may correctly select tools but hallucinate or incorrectly format parameter values, failing to reliably ground arguments in the source text. This results in structurally valid JSON that contains factually wrong data extracted from the conversation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}