How Needle 2 Handles Tool Calls with Structured JSON: A Complete Technical Guide

Needle 2 automatically converts Python functions into JSON Schema definitions, compiles them into deterministic byte-level grammars for the inference engine, and manages the full lifecycle of parsing, executing, and returning structured tool results to the model.

Needle 2 (from the cactus-compute/needle repository) implements a type-safe architecture for tool calling that bridges large language models and Python functions through strict JSON schemas. Unlike frameworks that rely on prompt engineering for structured output, Needle 2 generates deterministic grammars from Python type hints, forcing the model to emit valid JSON that conforms to predefined schemas at the token level.

The Three-Stage Pipeline for JSON Tool Handling

Stage 1: Schema Generation with @tool and build_schema

The foundation of Needle 2’s tool system lies in automatic JSON Schema generation. In needle/agent/tools.py, the @tool decorator inspects function signatures, type hints, and docstrings to produce a complete JSON Schema object ("type": "object") stored as the _needle_tool attribute on the function【/needle/agent/tools.py#L11-L42】.

The build_schema function handles complex type annotations, including:

• Standard types mapped to JSON Schema primitives
• Field constraints (enums, minimums, patterns) applied via Field.apply【/needle/agent/tools.py#L18-L33】
• Pydantic models recursively processed through pydantic_schema【/needle/agent/tools.py#L44-L61】

This schema includes parameter descriptions extracted from docstrings, default values, and validation constraints, creating a complete contract for the model to follow.

Stage 2: Native Engine Initialization and Grammar Compilation

When initializing a Needle instance in needle/__init__.py, the framework JSON-encodes all tool schemas into self._tools_json and passes them to the native needle_init C library function【/needle/init.py#L62-L66】. The native engine compiles these schemas into byte-level grammars that constrain the tokenizer during inference.

This compilation step ensures that every token generated for a tool call's "arguments" field must conform to the JSON Schema. The engine enforces type constraints, required fields, and validation rules at generation time, eliminating the possibility of malformed JSON or type mismatches in model outputs.

Stage 3: Invocation, Execution, and Result Handling

During conversation, Needle.run in needle/__init__.py orchestrates the tool execution loop【/needle/init.py#L94-L105】. The method:

1. Calls complete to receive the model's response
2. Detects "call" response types containing function_calls objects with tool names and JSON arguments
3. Looks up the corresponding Python callable via self._functions (populated during _resolve)
4. Executes the function with parsed arguments (fn(**call["arguments"]))
5. Captures return values or catches exceptions, returning errors as {"error": ...} structures【/needle/init.py#L34-L42】

Results are JSON-serialized and re-fed to the model via self.complete(json.dumps(results)), enabling multi-turn conversations where the model can reference tool outputs【/needle/init.py#L42-L45】.

Defining Tools with Structured JSON Schemas

The @tool decorator transforms standard Python functions into schema-backed tools. Type hints become JSON Schema types, while docstrings populate the description fields that guide the model’s understanding.

import needle
from needle import Field, tool

@tool  # Registers the function with a generated JSON schema

def get_weather(
    city: str,
    units: str = Field(default="metric", enum=["metric", "imperial"])
):
    """Get current weather conditions for a specified city."""
    # Implementation would call a weather API

    return {"city": city, "temperature": 22, "units": units}

The Field class adds JSON Schema constraints like enum, minimum, maximum, and pattern. In needle/agent/tools.py, these constraints merge into the final schema through Field.apply, ensuring the native engine enforces them during token generation.

Multi-Turn Execution with Needle.run

The Needle class manages the complete lifecycle of tool calls, from schema registration to result integration. When you invoke run, the framework handles the conversation loop automatically, detecting tool calls and executing them before returning the final response.

# Initialize with schema-backed tools

agent = needle.Needle(tools=[get_weather])

# The model receives the JSON schema, emits structured arguments,

# and Needle executes the Python function automatically

response = agent.run("What's the weather in Tokyo?")
print(response["results"])

# Output: [{'city': 'Tokyo', 'temperature': 22, 'units': 'metric'}]

In needle/__init__.py, the run method's implementation at lines 94-105 handles the transition between model generation and Python execution, maintaining conversation context across multiple tool invocations.

One-Shot Extraction with needle.extract

For structured data extraction without conversation, Needle 2 provides the extract function that treats a Pydantic model as a single-use tool. This shortcut uses the same schema generation pipeline but bypasses the multi-turn loop, returning parsed objects directly【/needle/init.py#L64-L77】.

from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"

# Schema is generated from Invoice model and executed once

invoice = needle.extract(text, Invoice)
print(invoice.vendor, invoice.total)

# Output: Acme Corp 1200.0

Summary

• Schema Generation: The @tool decorator in needle/agent/tools.py automatically produces JSON Schemas from Python functions using build_schema, storing them as _needle_tool attributes.
• Grammar Compilation: The Needle constructor passes JSON schemas to needle_init, where the C engine compiles deterministic grammars ensuring token-level JSON validity.
• Execution Loop: Needle.run manages the conversation cycle, parsing function_calls from model responses, executing Python functions with self._functions, and feeding results back to the model.
• Error Handling: Tool exceptions are captured and returned as structured {"error": ...} objects, allowing the model to handle failures gracefully.
• Extraction Mode: needle.extract provides single-shot structured output using the same schema pipeline optimized for direct data extraction.

Frequently Asked Questions

How does Needle 2 convert Python functions into JSON schemas?

Needle 2 uses the @tool decorator defined in needle/agent/tools.py to introspect function signatures via build_schema. It maps Python type hints to JSON Schema types, extracts descriptions from docstrings, and merges Field constraints (enums, ranges, patterns) into the schema object stored at fn._needle_tool.

What happens if a tool call fails or raises an exception?

When Needle.run executes a tool in needle/__init__.py, it wraps the function call in a try-except block【/needle/init.py#L34-L42】. Exceptions are captured and returned as JSON objects with an error key, allowing the model to receive structured failure information and potentially retry or request clarification.

Can I use Pydantic models directly as tools in Needle 2?

Yes. The pydantic_schema function in needle/agent/tools.py【/needle/agent/tools.py#L44-L61】 recursively processes Pydantic BaseModel classes into JSON Schema definitions. These can serve as parameters in tool functions or as standalone extraction targets via needle.extract, which treats the model as a single-turn tool call.

How does the native engine ensure valid JSON output?

During initialization in needle/__init__.py【/needle/init.py#L62-L66】, schemas are JSON-encoded and passed to the native needle_init function. The C engine compiles these into byte-level grammars that constrain the tokenizer during inference, guaranteeing that generated tool arguments conform exactly to the declared JSON Schema types and constraints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}