Fine-Tuning Needle: Complete Workflow from Data Synthesis to Model Export

Fine-tuning Needle involves preparing a JSON-L training dataset, optionally synthesizing additional examples via OpenRouter, training a LoRA adapter on the frozen base model, and exporting the merged weights to a portable .cact archive.

The cactus-compute/needle repository provides a streamlined pipeline for customizing tool-calling models. Whether you are augmenting existing data or training from scratch, the workflow centers on three core stages: data preparation, LoRA-based fine-tuning, and model export.

Preparing the Training Dataset

Needle expects training data in JSON-L format, where each line represents a single training example. According to the documentation in doc/finetuning.md (§13-17), every record must include:

• query: The user prompt or instruction
• schemas: The tool schema(s) the model should consider
• answers: The exact tool call(s) to be generated
• reasoning: (Optional) A grounding sentence explaining the tool selection

Each line in your data.jsonl file must be a valid JSON object containing these fields. The training loader in needle/model/finetune.py encodes these examples into token IDs and attention masks before feeding them into the model.

Synthesizing and Augmenting Training Data

When working with limited training data, Needle can generate synthetic examples using any OpenRouter-compatible LLM. The synthesis logic resides entirely in needle/model/finetune.py.

On-the-Fly Example Generation

The generate_examples() function (§86-96) constructs a prompt that enumerates your available tool schemas and instructs the LLM to produce valid JSON training examples. This function returns a batch of synthetic records matching the required JSON-L structure.

Parallel Dataset Generation

To scale up, generate_dataset() (§105-150) orchestrates multiple workers that repeatedly call generate_examples(). It deduplicates results across workers and terminates once the target sample count is reached, ensuring you receive a clean, non-redundant dataset.

Augmenting Existing Datasets

If you already have a seed dataset, augment_jsonl() (§64-76) reads your existing JSON-L file, extracts the tool schemas automatically, generates additional examples using the current diversity settings, and writes a combined output file. This preserves your original data while expanding coverage of edge cases.

CLI Command for Data Synthesis

Trigger the synthesis pipeline using the CLI:

# Generate 1,000 new examples and append to existing file

needle generate-data --augment data.jsonl --num-samples 1000

The resulting data.augmented.jsonl is ready for immediate training.

Training the LoRA Adapter

Needle fine-tunes a low-rank LoRA adapter while keeping the base model weights frozen. The training implementation lives in finetune_local() within needle/model/finetune.py.

Initialization and Tokenization

The trainer first loads the base checkpoint from checkpoints/needle2.pkl and initializes the tokenizer (§14-31). It calculates fit_max_len to determine the maximum sequence length for the batch, then encodes each JSON-L record into input token IDs and target masks (§26-58).

LoRA Configuration

The system creates LoRA weight groups via lora_target_paths and init_lora, then merges them into the base model using merge_lora during the forward pass. This approach minimizes memory footprint while allowing effective fine-tuning of tool-calling behavior.

Training Loop

The optimization loop (§87-127) runs cosine-annealed AdamW for the specified number of epochs. Each epoch reports training loss and validation loss, allowing you to monitor convergence without overfitting. Upon completion, the adapter is saved as a *.pkl file along with metadata required for export (§28-42).

Run training via CLI:

needle finetune data.jsonl --epochs 10 --out adapter.pkl

Exporting the Fine-Tuned Model

After training, you must merge the LoRA adapter back into the base weights to create a deployable artifact. The export logic is implemented in build_main() (§46-94) in needle/model/finetune.py.

Loading and Merging

build_main() loads the base checkpoint from checkpoints/needle2.pkl and optionally loads your trained LoRA adapter (§46-62). If quantization-aware training (QAT) metadata is present, it applies those transformations to maintain inference efficiency (§66-82).

Creating the Deployment Archive

Finally, write_export() serializes the merged weights into a .cact archive (§88-94). This portable file contains everything needed to instantiate your fine-tuned agent.

Export via CLI:

needle build checkpoints/needle2.pkl --lora adapter.pkl --out tuned.cact

Load the model in Python:

import needle
agent = needle.Needle(tools=[...], weights="tuned.cact")

Complete CLI Workflow Example

The entire pipeline from data augmentation to deployment follows this sequence:

# Step 1: Generate synthetic training data

needle generate-data --augment data.jsonl --num-samples 1000

# Step 2: Fine-tune the LoRA adapter

needle finetune data.augmented.jsonl --epochs 20 --out adapter.pkl

# Step 3: Export the merged model

needle build checkpoints/needle2.pkl --lora adapter.pkl --out tuned.cact

# Step 4: Deploy in Python

python -c "import needle; agent = needle.Needle(tools=[...], weights='tuned.cact')"

Summary

• Data format: Needle requires JSON-L files containing queries, tool schemas, and target answers, with optional reasoning strings.
• Synthesis: Use generate_examples() and augment_jsonl() in needle/model/finetune.py, or run needle generate-data to create synthetic training data via OpenRouter.
• Training: The finetune_local() function trains a LoRA adapter using cosine-annealed AdamW while keeping the base model from checkpoints/needle2.pkl frozen.
• Export: build_main() merges the LoRA weights and exports a .cact archive via needle build, ready for inference.

Frequently Asked Questions

What is the exact format for Needle training data?

Each line in your JSON-L file must be a JSON object with query, schemas, and answers keys. The reasoning field is optional but recommended for complex tool chains. The schema must match the format described in doc/finetuning.md (§13-17), where answers contains the exact tool calls the model should learn to generate.

Can I use synthetic data for production fine-tuning?

Yes. The generate_dataset() function in needle/model/finetune.py (§105-150) produces deduplicated, schema-validated examples. However, best results typically come from mixing synthetic data with real user queries to ensure the model handles actual production distributions, not just synthetic variations.

Why does Needle use LoRA instead of full fine-tuning?

Needle implements LoRA (Low-Rank Adaptation) to minimize computational requirements and prevent catastrophic forgetting. By training only the low-rank adapter matrices while keeping the base weights from checkpoints/needle2.pkl frozen, you can fine-tune on consumer hardware and merge the adapter later via build_main() without degrading the base model's general capabilities.

What is the difference between the .pkl and .cact files?

The .pkl file contains only the LoRA adapter weights and training metadata saved by finetune_local(). The .cact file is the final deployment artifact created by needle build via write_export(); it contains the fully merged base model plus adapter weights, ready to load directly into needle.Needle().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}