# How to Synthesize Training Data with OpenRouter for Needle: Dataset Size and Generation Guide

> Synthesize training data for Needle with OpenRouter. Learn dataset size requirements for tool selection and argument grounding. Generate hundreds to thousands of samples easily.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-18

---

**You can synthesize training data for Needle by running `needle generate-data` with an `OPENROUTER_API_KEY`, providing a tool schema JSON file, and requesting hundreds to thousands of samples depending on whether you are training for tool selection (300–500 samples) or argument grounding (1,000–5,000 samples).**

Needle is an open-source framework (cactus-compute/needle) for fine-tuning language models to reliably invoke tools. To teach the model which function to call and how to populate its parameters, you must furnish a JSONL dataset of user queries paired with correct tool invocations. The most efficient way to build this corpus is to synthesize training data with OpenRouter using Needle's built-in generator, which converts tool schemas into realistic conversational examples without manual labeling.

## Configuring OpenRouter Authentication

Before invoking the generator, export your API credentials. The `generate-data` command expects an `OPENROUTER_API_KEY` environment variable, and optionally `OPENROUTER_URL` if you are routing through a custom gateway.

```bash
export OPENROUTER_API_KEY=sk-or-...

# Optional: export OPENROUTER_URL=https://custom.gateway/v1

```

## Generating Synthetic Training Data with `generate-data`

The `generate-data` CLI command contacts the OpenRouter API (or any OpenAI-compatible endpoint), reads your tool definitions, and emits synthetic query-tool pairs. According to the [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) (lines 87-93), this is the primary mechanism for populating your training corpus.

### Creating a Dataset from Scratch

Point `--tools` at a JSON schema describing your function catalogue, set `--num-samples` to your desired volume, and specify `--output` for the destination JSONL file.

```bash
needle generate-data \
  --tools my_tools.json \
  --num-samples 800 \
  --output training_data.jsonl

```

The generator explores linguistic variations around your tool definitions, producing diverse phrasings and realistic argument values.

### Augmenting Existing Seed Data

If you already possess a hand-crafted dataset, expand it with the `--augment` flag. This appends new synthetic examples to your existing file rather than overwriting it.

```bash
needle generate-data \
  --augment existing_data.jsonl \
  --num-samples 500

```

## Dataset Size Requirements for Needle Fine-Tuning

Not all training objectives require the same volume of data. As documented in [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md) (lines 68-71), Needle distinguishes between **tool selection** (picking the correct function) and **argument grounding** (correctly filling parameters).

### Tool Selection vs. Argument Grounding

- **Tool selection** improves rapidly with a few hundred high-quality examples. The model learns to map user intent to the appropriate function name.
- **Argument grounding** requires **thousands** of varied examples. The model must learn to extract specific values from the query text and map them to the correct JSON fields, often captured in the optional `reasoning` field.

### Recommended Sample Counts by Catalog Size

Scale your dataset according to the complexity of your tool catalogue:

- **Small catalogues (≤10 tools):**
  - 300–500 synthetic samples for reliable tool selection
  - 1,000–2,000 additional samples for robust argument grounding

- **Large catalogues (≥20 tools):**
  - 1,000–2,000 samples for selection
  - 3,000–5,000 samples for grounding

For large catalogues, consider splitting the process into two inference passes: first train for full-catalogue selection, then train a separate model (or second phase) for single-tool grounding.

## Data Format and Off-Topic Examples

Each line in the output JSONL contains `query`, `tools`, `answers`, and an optional `reasoning` field. The `answers` array holds the target tool calls.

Include a subset of off-topic examples where `"answers": []`. These teach the model to abstain from tool invocation when the user query is irrelevant to the available tools.

Example generated record:

```json
{
  "query": "Dim the kitchen lights to 10 percent",
  "tools": [
    {
      "name": "set_lights",
      "description": "Set a room's light brightness.",
      "parameters": {
        "type": "object",
        "properties": {
          "room": {"type": "string"},
          "brightness": {"type": "integer"}
        },
        "required": ["room", "brightness"]
      }
    }
  ],
  "answers": [
    {
      "name": "set_lights",
      "arguments": {"room": "kitchen", "brightness": 10}
    }
  ],
  "reasoning": "'kitchen' → room; '10' → brightness"
}

```

## Summary

- **Set credentials**: Export `OPENROUTER_API_KEY` before running Needle's generator.
- **Generate data**: Use `needle generate-data --tools schema.json --num-samples N` to create synthetic training pairs.
- **Augment seeds**: Append to existing datasets with the `--augment` flag.
- **Size appropriately**: Allocate 300–500 samples for tool selection and 1,000–5,000 for argument grounding, scaling with catalogue size.
- **Include negatives**: Add examples with empty `answers` arrays to teach abstention.

## Frequently Asked Questions

### Do I need to manually label data to train Needle?

No. While you can hand-craft examples, the `generate-data` command automates synthesis via OpenRouter, producing varied, realistic query-tool pairs from your JSON schema definitions. This eliminates the need for manual annotation while maintaining high data quality.

### Can I use a different LLM provider instead of OpenRouter?

Yes. Export `OPENROUTER_URL` to point at any OpenAI-compatible API gateway. The generator uses standard chat completions, so compatible endpoints like OpenAI, Groq, or local vLLM instances work interchangeably with the same `generate-data` interface.

### How does Needle use the `reasoning` field in training data?

The optional `reasoning` field documents how argument values map to spans in the user query. Including this textual explanation improves the model's argument grounding accuracy by providing explicit supervision for parameter extraction, showing the model exactly which words in the query correspond to each JSON value.

### What happens if I provide too few samples for argument grounding?

With insufficient examples (fewer than 1,000 for small catalogues), the model may correctly select tools but hallucinate or incorrectly format parameter values, failing to reliably ground arguments in the source text. This results in structurally valid JSON that contains factually wrong data extracted from the conversation.