Needle 2 Capabilities: Tool Calling, Device Use, and Structured Extraction Explained

Needle 2 offers tool calling, structured extraction, and model fine-tuning through a lightweight LLM-agent framework that converts Python functions into JSON Schema tools and parses Pydantic models via special tokenizer markup.

Needle 2, developed in the cactus-compute/needle repository, gives developers a minimal yet powerful way to turn language models into agents that can invoke Python functions and return typed data objects. Its core Needle 2 capabilities revolve around two programmable devices: a tool-calling device that executes arbitrary functions via <tool_call> tokens, and a structured-extraction device that maps free-form generation into Pydantic models. These features are implemented across a small, focused codebase with explicit schema generation and deterministic token parsing.

How Tool Calling Works in Needle 2

Needle 2 turns ordinary Python functions into LLM-invokable tools using schema-driven markup and a closed inference loop.

The @tool Decorator and JSON Schema Generation

In needle/agent/tools.py, the @tool decorator introspects a function’s signature and docstring to build a JSON Schema description of its name, description, and parameters. This schema is stored on the function object itself as fn._needle_tool via the internal build_schema logic. When you instantiate a Needle agent with a list of decorated functions, that schema list is injected directly into the model’s context.

Special Tokens for Tool Markup

The framework keeps tool-related context deterministic by defining custom tokens in needle/model/tokenizer.py. These include <tools>, </tools>, <tool_call>, </tool_call>, <tool_result>, and </tool_result>. By delimiting tool definitions, calls, and results with explicit tokens, Needle avoids ambiguous parsing and keeps the generation loop tidy.

End-to-End Inference Loop

The Needle class in needle/__init__.py orchestrates the full lifecycle: it loads the model, injects tool schemas, generates text, detects <tool_call> tags in the output, executes the matching Python function, and feeds the wrapped <tool_result> back into the next generation step. This loop repeats until the model produces a final answer without requesting further tool use.

Structured Extraction in Needle 2

Beyond free-form text, Needle 2 can force model output into strongly-typed Python objects using Pydantic.

Automatic JSON Schema from Pydantic Models

In needle/agent/tools.py, the pydantic_schema utility and _is_pydantic_model helper convert any BaseModel or enum.Enum into a JSON Schema automatically. When you pass a Pydantic model to the agent, Needle prompts the model to fill that schema instead of emitting unstructured prose.

Typed Output Parsing

After the model returns JSON that conforms to the schema, Needle parses it back into the original Python type. This means a function annotated to return a Pydantic model can serve as both a tool and a structured extractor, letting the agent hand back native Python objects rather than raw strings.

Fine-Tuning and Model Utilities

Needle 2 ships with optional pipelines that let you teach base models to use these devices more reliably, plus utilities for efficient deployment.

Fine-Tuning for Tool Calling

The needle/model/finetune.py module provides generate_examples and _collect_tools to create OpenAI-style JSON-L training data. The pipeline deduplicates tool schemas, synthesizes examples containing valid <tool_call> markup, and writes them in a format ready for supervised fine-tuning.

Quantization and Export

For production inference, needle/model/quantize.py supports int8 and float16 compression, while needle/model/export.py handles portable model serialization. These are fully optional but let you shrink model size without stripping away tool-calling behavior.

Practical Code Examples

The snippets below demonstrate real-world usage of the tool-calling and structured-extraction devices.

Calling a Tool from the Model

from needle import tool, Needle

@tool
def send_email(to: str, subject: str, body: str) -> str:
    """Send an email and return the message ID."""
    return "msg-12345"

agent = Needle(tools=[send_email])

response = agent.complete(
    tools_json='[{"name": "send_email", "parameters": {"type": "object", "properties": {"to": {"type":"string"}, "subject":{"type":"string"}, "body":{"type":"string"}}}}]',
    query="Email Alice <alice@example.com> saying: 'Project is on track.'"
)
print(response)

# The model emits <tool_call> for send_email,

# Needle executes it, injects <tool_result>, and continues.

Extracting Structured Data with Pydantic

from pydantic import BaseModel
from needle import Needle, tool

class WeatherReport(BaseModel):
    """Current weather conditions."""
    location: str
    temperature_c: float
    condition: str

@tool
def get_weather(city: str) -> WeatherReport:
    """Fetch weather for *city* and return a WeatherReport."""
    return WeatherReport(location=city, temperature_c=22.5, condition="Sunny")

agent = Needle(tools=[get_weather])

response = agent.complete(
    tools_json='[{"name": "get_weather", "parameters": {"type":"object","properties":{"city":{"type":"string"}}}}]',
    query="Give me the weather in Paris."
)
print(response)

# The model returns JSON matching WeatherReport,

# which Needle parses back into a WeatherReport instance.

Summary

• Needle 2 capabilities center on a tool-calling device and a structured-extraction device that turn LLM output into executable actions and typed objects.
• The @tool decorator in needle/agent/tools.py generates JSON Schema metadata and attaches it to functions as fn._needle_tool.
• Special tokens in needle/model/tokenizer.py delimit tool definitions and results for deterministic parsing.
• The Needle class in needle/__init__.py runs an inference loop that detects <tool_call> tokens, executes Python functions, and feeds <tool_result> back into context.
• Structured extraction relies on pydantic_schema in needle/agent/tools.py to convert Pydantic models into JSON Schema and parse model output back into native Python types.
• Optional fine-tuning utilities in needle/model/finetune.py let you train models to emit tool markup, while needle/model/quantize.py and needle/model/export.py support compressed, portable deployments.

Frequently Asked Questions

What does the @tool decorator do in Needle 2?

The @tool decorator registers a Python function as an LLM-invokable tool by building a JSON Schema of its parameters and storing that schema on fn._needle_tool in needle/agent/tools.py. This allows the Needle agent to pass the function signature to the model and recognize when the model requests its execution via <tool_call> tokens.

How does Needle 2 return structured data instead of plain text?

Needle 2 uses the pydantic_schema utility in needle/agent/tools.py to convert a Pydantic BaseModel into a JSON Schema prompt. The model is then guided to fill that schema with valid JSON, which Needle parses back into the original Python type, enabling strongly-typed structured extraction.

Can I fine-tune a base model to use Needle 2 tool calling?

Yes. The needle/model/finetune.py module includes generate_examples and _collect_tools, which generate deduplicated, synthetic training examples in JSON-L format. These examples teach a base model to emit <tool_call> markup and structured JSON, making the tool-calling device more reliable after fine-tuning.

What special tokens does Needle 2 use during tool calling?

According to needle/model/tokenizer.py, Needle 2 reserves <tools>, </tools>, <tool_call>, </tool_call>, <tool_result>, and </tool_result> to wrap tool schemas, model requests, and function outputs. This unified tokenization keeps parsing deterministic and separates tool logic from regular generation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}