How to Use the Needle Agent for Tool Calling and Structured Extraction

The Needle Agent is a lightweight Python wrapper around the native C++ Needle engine that enables LLMs to invoke arbitrary Python functions during generation and extract structured data into validated Pydantic models through a unified tool-calling architecture.

The Needle Agent provides a high-level interface for integrating tool use and schema validation into LLM workflows. As implemented in cactus-compute/needle, the agent orchestrates the request-response cycle between Python callables and the native engine, supporting both interactive tool invocation and one-shot structured extraction through three core components defined in needle/__init__.py, needle/agent/tools.py, and needle/agent/fetch.py.

Needle Agent Architecture Overview

The agent operates through a three-component system that bridges Python and the underlying C++ engine:

• Needle class (needle/__init__.py): Manages the C library handle, loads optional fine-tuned weights, and orchestrates the request-response cycle. It maintains the self._functions registry and implements the run() loop that dispatches tool calls.

• Tool registration (needle/agent/tools.py): Implements the @tool decorator and build_schema() function. The decorator attaches a pre-computed JSON schema to callables, while build_schema() auto-generates schemas from type hints for undecorated functions.

• Extraction helper (needle/__init__.py, lines 30-55): Provides the extract() function for one-shot structured data retrieval, creating temporary agent instances with Pydantic models as the sole tool.

Tool Calling with the Needle Agent

Tool calling enables the LLM to invoke Python functions during text generation. The implementation stores tool schemas in self._functions and executes a multi-step dispatch loop.

Defining Tools with the @tool Decorator

Tools are defined using the @tool decorator or raw type hints. The decorator, implemented in needle/agent/tools.py, attaches a JSON schema to the function object:

# needle/agent/tools.py

def tool(fn: Callable) -> Callable:
    fn._needle_tool = build_schema(fn)
    return fn

Alternatively, Needle auto-generates schemas via build_schema() when undecorated functions are passed to the tools argument.

The Tool Execution Loop

The execution flow follows five distinct phases:

1. Schema resolution: During initialization, Needle._resolve (lines 38-49 in needle/__init__.py) converts each tool into a JSON schema and populates self._functions.

2. Prompt submission: The run() method sends the user query to the engine via _complete().

3. Function dispatch: When the engine returns a "type": "call" payload, Needle.run iterates over function_calls, looks up each callable in self._functions, and executes it with the provided arguments (lines 84-104):

# needle/__init__.py (simplified from lines 84-104)

for call in calls:
    fn = self._functions.get(call.get("name"))
    results.append(fn(**(call.get("arguments") or {})))
response = self._complete(json.dumps(results, default=_jsonable), max_new_tokens)

4. Result feedback: Serialized results are fed back to the engine for subsequent reasoning steps, looping up to max_steps.

5. Final aggregation: The accumulated executed list is attached as response["results"] and returned to the caller.

Structured Extraction from Text

Structured extraction leverages the same underlying architecture to parse unstructured text into typed Python objects, typically Pydantic models.

One-Shot Extraction with Pydantic Models

The high-level extract() function (lines 30-55 in needle/__init__.py) creates a temporary agent instance using the supplied schema as the sole available tool:

# needle/__init__.py (lines 30-55)

agent = Needle(tools=[schema], system=system, weights=selected)
response = agent._complete(text, max_new_tokens)
arguments = calls[0].get("arguments") or {}
if strict:
    _validate_extraction(text, schema, arguments, response)
return schema(**arguments) if _is_pydantic_model(schema) else arguments

The function parses the first function_calls entry to obtain arguments, optionally validates the extraction, and returns either a Pydantic model instance or a plain dictionary.

Validation and Error Handling

When strict=True, the _validate_extraction() helper performs temporal grounding checks and negation flag validation. If the extracted arguments contradict temporal constraints or negation markers present in the source text, the function raises ExtractionValidationError.

Setting Up the Native Engine

Before instantiating the Needle class, the C++ engine binaries must be fetched from Hugging Face. The fetch_library() function in needle/agent/fetch.py (lines 13-31) downloads platform-specific shared libraries (libneedle.so, .dylib, or .dll) and caches them under ~/.cache/cactus-needle:

# needle/agent/fetch.py (lines 13-31)

path = hf_hub_download(..., filename="python/" + wheel, repo_type="model")
with zipfile.ZipFile(path) as archive:
    data = archive.read("needle/" + lib)

This fetch step is mandatory; any Needle instantiation will fail if the native library is not present in the cache directory.

Practical Examples

Basic Tool Calling Example

This example demonstrates registering a Python function and invoking it through the agent:

from needle import Needle, tool

@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

needle = Needle(tools=[add])
resp = needle.run(
    query="What is the sum of 7 and 12? Use the add tool.",
    max_steps=2,
)
print(resp["results"])

# → [{'result': 19}]

The @tool decorator attaches the schema metadata in needle/agent/tools.py, while the execution loop resides in needle/__init__.py lines 84-104.

Structured Data Extraction Example

Extract typed events from unstructured text using Pydantic validation:

from pydantic import BaseModel, Field
from needle import extract

class Event(BaseModel):
    title: str
    date: int = Field(..., description="Year of the event")
    location: str | None = None

text = "The conference called PyCon will happen in 2025 in Boston."
event = extract(text, Event, strict=True)
print(event)

# → title='PyCon' date=2025 location='Boston'

This uses the extract implementation at lines 30-55 of needle/__init__.py and the _validate_extraction helper for strict mode verification.

Command-Line Interface Usage

The CLI exposes tool-enabled queries without Python boilerplate:

needle run \
  --checkpoint my-model.cact \
  --query "Find the latest release version on GitHub." \
  --tools '[{"name":"fetch_github_release","description":"Fetch latest tag","parameters":{"type":"object","properties":{"repo":{"type":"string"}}}}]'

The CLI parses the JSON tool definitions in needle/cli.py (lines 5-30) and constructs the Needle instance internally.

Summary

• The Needle Agent wraps the C++ engine to provide Python-native tool calling and structured extraction capabilities.
• Tools are registered via the @tool decorator or auto-generated schemas from type hints, stored in self._functions, and dispatched through the run() loop in needle/__init__.py.
• Structured extraction uses the extract() function to create temporary agents with Pydantic models, validating outputs against temporal and negation constraints when strict=True.
• The native engine is fetched automatically from Hugging Face into ~/.cache/cactus-needle via needle/agent/fetch.py.
• Both capabilities share the same underlying architecture, differing only in execution duration (multi-step conversation vs. one-shot completion).

Frequently Asked Questions

How does the Needle Agent handle tool schema generation for undecorated functions?

According to the source code in needle/agent/tools.py, undecorated functions passed to the Needle constructor are processed by build_schema(), which inspects type hints and docstrings to generate JSON schemas dynamically. This allows plain Python functions to be used as tools without requiring the @tool decorator, though explicit decoration provides more control over schema metadata.

What validation occurs when using strict=True in structured extraction?

As implemented in needle/__init__.py, the _validate_extraction() helper checks for temporal grounding consistency and negation flags between the source text and extracted arguments. If the extracted values contradict temporal markers (e.g., past vs. future dates) or improperly handle negated statements in the input, the function raises ExtractionValidationError before returning the typed object.

Can the Needle Agent execute multiple tool calls in a single generation step?

Yes. The run() method in needle/__init__.py (lines 84-104) iterates over all function_calls returned by the engine in a single response, executing each callable sequentially and collecting results. These results are serialized and fed back to the engine in a subsequent completion request, supporting multi-step reasoning workflows up to the specified max_steps limit.

Where does the Needle Agent store downloaded engine binaries and how are they loaded?

The fetch_library() function in needle/agent/fetch.py downloads platform-specific wheels from Hugging Face, extracts the shared library (libneedle.so, .dylib, or .dll), and stores it under ~/.cache/cactus-needle. The Needle class in needle/__init__.py loads this library at instantiation time; if the binary is missing, instantiation will fail with an error indicating that fetch_library() must be called first.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}