# Needle 2 Capabilities: Tool Calling, Device Use, and Structured Extraction Explained

> Explore Needle 2's capabilities: tool calling, device use, and structured extraction. Discover how this LLM-agent framework converts Python functions to JSON Schema tools and parses Pydantic models efficiently.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-19

---

**Needle 2 offers tool calling, structured extraction, and model fine-tuning through a lightweight LLM-agent framework that converts Python functions into JSON Schema tools and parses Pydantic models via special tokenizer markup.**

Needle 2, developed in the `cactus-compute/needle` repository, gives developers a minimal yet powerful way to turn language models into agents that can invoke Python functions and return typed data objects. Its core **Needle 2 capabilities** revolve around two programmable devices: a tool-calling device that executes arbitrary functions via `<tool_call>` tokens, and a structured-extraction device that maps free-form generation into Pydantic models. These features are implemented across a small, focused codebase with explicit schema generation and deterministic token parsing.

## How Tool Calling Works in Needle 2

Needle 2 turns ordinary Python functions into LLM-invokable tools using schema-driven markup and a closed inference loop.

### The `@tool` Decorator and JSON Schema Generation

In [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), the `@tool` decorator introspects a function’s signature and docstring to build a JSON Schema description of its name, description, and parameters. This schema is stored on the function object itself as `fn._needle_tool` via the internal `build_schema` logic. When you instantiate a `Needle` agent with a list of decorated functions, that schema list is injected directly into the model’s context.

### Special Tokens for Tool Markup

The framework keeps tool-related context deterministic by defining custom tokens in [`needle/model/tokenizer.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/tokenizer.py). These include `<tools>`, `</tools>`, `<tool_call>`, `</tool_call>`, `<tool_result>`, and `</tool_result>`. By delimiting tool definitions, calls, and results with explicit tokens, Needle avoids ambiguous parsing and keeps the generation loop tidy.

### End-to-End Inference Loop

The `Needle` class in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) orchestrates the full lifecycle: it loads the model, injects tool schemas, generates text, detects `<tool_call>` tags in the output, executes the matching Python function, and feeds the wrapped `<tool_result>` back into the next generation step. This loop repeats until the model produces a final answer without requesting further tool use.

## Structured Extraction in Needle 2

Beyond free-form text, Needle 2 can force model output into strongly-typed Python objects using Pydantic.

### Automatic JSON Schema from Pydantic Models

In [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), the `pydantic_schema` utility and `_is_pydantic_model` helper convert any `BaseModel` or `enum.Enum` into a JSON Schema automatically. When you pass a Pydantic model to the agent, Needle prompts the model to fill that schema instead of emitting unstructured prose.

### Typed Output Parsing

After the model returns JSON that conforms to the schema, Needle parses it back into the original Python type. This means a function annotated to return a Pydantic model can serve as both a tool and a structured extractor, letting the agent hand back native Python objects rather than raw strings.

## Fine-Tuning and Model Utilities

Needle 2 ships with optional pipelines that let you teach base models to use these devices more reliably, plus utilities for efficient deployment.

### Fine-Tuning for Tool Calling

The [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) module provides `generate_examples` and `_collect_tools` to create OpenAI-style JSON-L training data. The pipeline deduplicates tool schemas, synthesizes examples containing valid `<tool_call>` markup, and writes them in a format ready for supervised fine-tuning.

### Quantization and Export

For production inference, [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) supports int8 and float16 compression, while [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) handles portable model serialization. These are fully optional but let you shrink model size without stripping away tool-calling behavior.

## Practical Code Examples

The snippets below demonstrate real-world usage of the tool-calling and structured-extraction devices.

### Calling a Tool from the Model

```python
from needle import tool, Needle

@tool
def send_email(to: str, subject: str, body: str) -> str:
    """Send an email and return the message ID."""
    return "msg-12345"

agent = Needle(tools=[send_email])

response = agent.complete(
    tools_json='[{"name": "send_email", "parameters": {"type": "object", "properties": {"to": {"type":"string"}, "subject":{"type":"string"}, "body":{"type":"string"}}}}]',
    query="Email Alice <alice@example.com> saying: 'Project is on track.'"
)
print(response)

# The model emits <tool_call> for send_email,

# Needle executes it, injects <tool_result>, and continues.

```

### Extracting Structured Data with Pydantic

```python
from pydantic import BaseModel
from needle import Needle, tool

class WeatherReport(BaseModel):
    """Current weather conditions."""
    location: str
    temperature_c: float
    condition: str

@tool
def get_weather(city: str) -> WeatherReport:
    """Fetch weather for *city* and return a WeatherReport."""
    return WeatherReport(location=city, temperature_c=22.5, condition="Sunny")

agent = Needle(tools=[get_weather])

response = agent.complete(
    tools_json='[{"name": "get_weather", "parameters": {"type":"object","properties":{"city":{"type":"string"}}}}]',
    query="Give me the weather in Paris."
)
print(response)

# The model returns JSON matching WeatherReport,

# which Needle parses back into a WeatherReport instance.

```

## Summary

- Needle 2 capabilities center on a **tool-calling device** and a **structured-extraction device** that turn LLM output into executable actions and typed objects.
- The `@tool` decorator in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) generates JSON Schema metadata and attaches it to functions as `fn._needle_tool`.
- Special tokens in [`needle/model/tokenizer.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/tokenizer.py) delimit tool definitions and results for deterministic parsing.
- The `Needle` class in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) runs an inference loop that detects `<tool_call>` tokens, executes Python functions, and feeds `<tool_result>` back into context.
- Structured extraction relies on `pydantic_schema` in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) to convert Pydantic models into JSON Schema and parse model output back into native Python types.
- Optional fine-tuning utilities in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) let you train models to emit tool markup, while [`needle/model/quantize.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/quantize.py) and [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) support compressed, portable deployments.

## Frequently Asked Questions

### What does the `@tool` decorator do in Needle 2?

The `@tool` decorator registers a Python function as an LLM-invokable tool by building a JSON Schema of its parameters and storing that schema on `fn._needle_tool` in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py). This allows the `Needle` agent to pass the function signature to the model and recognize when the model requests its execution via `<tool_call>` tokens.

### How does Needle 2 return structured data instead of plain text?

Needle 2 uses the `pydantic_schema` utility in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) to convert a Pydantic `BaseModel` into a JSON Schema prompt. The model is then guided to fill that schema with valid JSON, which Needle parses back into the original Python type, enabling strongly-typed structured extraction.

### Can I fine-tune a base model to use Needle 2 tool calling?

Yes. The [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) module includes `generate_examples` and `_collect_tools`, which generate deduplicated, synthetic training examples in JSON-L format. These examples teach a base model to emit `<tool_call>` markup and structured JSON, making the tool-calling device more reliable after fine-tuning.

### What special tokens does Needle 2 use during tool calling?

According to [`needle/model/tokenizer.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/tokenizer.py), Needle 2 reserves `<tools>`, `</tools>`, `<tool_call>`, `</tool_call>`, `<tool_result>`, and `</tool_result>` to wrap tool schemas, model requests, and function outputs. This unified tokenization keeps parsing deterministic and separates tool logic from regular generation.