# How Needle 2 Handles Tool Calls with Structured JSON: A Complete Technical Guide

> Learn how Needle 2 handles tool calls with structured JSON. Discover its automatic conversion of Python functions to JSON Schema and efficient tool result management.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-17

---

**Needle 2 automatically converts Python functions into JSON Schema definitions, compiles them into deterministic byte-level grammars for the inference engine, and manages the full lifecycle of parsing, executing, and returning structured tool results to the model.**

Needle 2 (from the `cactus-compute/needle` repository) implements a type-safe architecture for tool calling that bridges large language models and Python functions through strict JSON schemas. Unlike frameworks that rely on prompt engineering for structured output, Needle 2 generates deterministic grammars from Python type hints, forcing the model to emit valid JSON that conforms to predefined schemas at the token level.

## The Three-Stage Pipeline for JSON Tool Handling

### Stage 1: Schema Generation with `@tool` and `build_schema`

The foundation of Needle 2’s tool system lies in automatic JSON Schema generation. In [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), the `@tool` decorator inspects function signatures, type hints, and docstrings to produce a complete JSON Schema object (`"type": "object"`) stored as the `_needle_tool` attribute on the function【/needle/agent/tools.py#L11-L42】.

The `build_schema` function handles complex type annotations, including:
- **Standard types** mapped to JSON Schema primitives
- **`Field` constraints** (enums, minimums, patterns) applied via `Field.apply`【/needle/agent/tools.py#L18-L33】
- **Pydantic models** recursively processed through `pydantic_schema`【/needle/agent/tools.py#L44-L61】

This schema includes parameter descriptions extracted from docstrings, default values, and validation constraints, creating a complete contract for the model to follow.

### Stage 2: Native Engine Initialization and Grammar Compilation

When initializing a `Needle` instance in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), the framework JSON-encodes all tool schemas into `self._tools_json` and passes them to the native `needle_init` C library function【/needle/__init__.py#L62-L66】. The native engine compiles these schemas into **byte-level grammars** that constrain the tokenizer during inference.

This compilation step ensures that every token generated for a tool call's `"arguments"` field must conform to the JSON Schema. The engine enforces type constraints, required fields, and validation rules at generation time, eliminating the possibility of malformed JSON or type mismatches in model outputs.

### Stage 3: Invocation, Execution, and Result Handling

During conversation, `Needle.run` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) orchestrates the tool execution loop【/needle/__init__.py#L94-L105】. The method:
1. Calls `complete` to receive the model's response
2. Detects `"call"` response types containing `function_calls` objects with tool names and JSON arguments
3. Looks up the corresponding Python callable via `self._functions` (populated during `_resolve`)
4. Executes the function with parsed arguments (`fn(**call["arguments"])`)
5. Captures return values or catches exceptions, returning errors as `{"error": ...}` structures【/needle/__init__.py#L34-L42】

Results are JSON-serialized and re-fed to the model via `self.complete(json.dumps(results))`, enabling multi-turn conversations where the model can reference tool outputs【/needle/__init__.py#L42-L45】.

## Defining Tools with Structured JSON Schemas

The `@tool` decorator transforms standard Python functions into schema-backed tools. Type hints become JSON Schema types, while docstrings populate the description fields that guide the model’s understanding.

```python
import needle
from needle import Field, tool

@tool  # Registers the function with a generated JSON schema

def get_weather(
    city: str,
    units: str = Field(default="metric", enum=["metric", "imperial"])
):
    """Get current weather conditions for a specified city."""
    # Implementation would call a weather API

    return {"city": city, "temperature": 22, "units": units}

```

The `Field` class adds JSON Schema constraints like `enum`, `minimum`, `maximum`, and `pattern`. In [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), these constraints merge into the final schema through `Field.apply`, ensuring the native engine enforces them during token generation.

## Multi-Turn Execution with `Needle.run`

The `Needle` class manages the complete lifecycle of tool calls, from schema registration to result integration. When you invoke `run`, the framework handles the conversation loop automatically, detecting tool calls and executing them before returning the final response.

```python

# Initialize with schema-backed tools

agent = needle.Needle(tools=[get_weather])

# The model receives the JSON schema, emits structured arguments,

# and Needle executes the Python function automatically

response = agent.run("What's the weather in Tokyo?")
print(response["results"])

# Output: [{'city': 'Tokyo', 'temperature': 22, 'units': 'metric'}]

```

In [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), the `run` method's implementation at lines 94-105 handles the transition between model generation and Python execution, maintaining conversation context across multiple tool invocations.

## One-Shot Extraction with `needle.extract`

For structured data extraction without conversation, Needle 2 provides the `extract` function that treats a Pydantic model as a single-use tool. This shortcut uses the same schema generation pipeline but bypasses the multi-turn loop, returning parsed objects directly【/needle/__init__.py#L64-L77】.

```python
from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"

# Schema is generated from Invoice model and executed once

invoice = needle.extract(text, Invoice)
print(invoice.vendor, invoice.total)

# Output: Acme Corp 1200.0

```

## Summary

- **Schema Generation**: The `@tool` decorator in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) automatically produces JSON Schemas from Python functions using `build_schema`, storing them as `_needle_tool` attributes.
- **Grammar Compilation**: The `Needle` constructor passes JSON schemas to `needle_init`, where the C engine compiles deterministic grammars ensuring token-level JSON validity.
- **Execution Loop**: `Needle.run` manages the conversation cycle, parsing `function_calls` from model responses, executing Python functions with `self._functions`, and feeding results back to the model.
- **Error Handling**: Tool exceptions are captured and returned as structured `{"error": ...}` objects, allowing the model to handle failures gracefully.
- **Extraction Mode**: `needle.extract` provides single-shot structured output using the same schema pipeline optimized for direct data extraction.

## Frequently Asked Questions

### How does Needle 2 convert Python functions into JSON schemas?

Needle 2 uses the `@tool` decorator defined in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) to introspect function signatures via `build_schema`. It maps Python type hints to JSON Schema types, extracts descriptions from docstrings, and merges `Field` constraints (enums, ranges, patterns) into the schema object stored at `fn._needle_tool`.

### What happens if a tool call fails or raises an exception?

When `Needle.run` executes a tool in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), it wraps the function call in a try-except block【/needle/__init__.py#L34-L42】. Exceptions are captured and returned as JSON objects with an `error` key, allowing the model to receive structured failure information and potentially retry or request clarification.

### Can I use Pydantic models directly as tools in Needle 2?

Yes. The `pydantic_schema` function in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)【/needle/agent/tools.py#L44-L61】 recursively processes Pydantic `BaseModel` classes into JSON Schema definitions. These can serve as parameters in tool functions or as standalone extraction targets via `needle.extract`, which treats the model as a single-turn tool call.

### How does the native engine ensure valid JSON output?

During initialization in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)【/needle/__init__.py#L62-L66】, schemas are JSON-encoded and passed to the native `needle_init` function. The C engine compiles these into byte-level grammars that constrain the tokenizer during inference, guaranteeing that generated tool arguments conform exactly to the declared JSON Schema types and constraints.