# How to Use the Needle Agent for Tool Calling and Structured Extraction

> Learn to use the Needle Agent for LLM tool calling and structured data extraction. Integrate Python functions and Pydantic models seamlessly with this powerful wrapper.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-04

---

**The Needle Agent is a lightweight Python wrapper around the native C++ Needle engine that enables LLMs to invoke arbitrary Python functions during generation and extract structured data into validated Pydantic models through a unified tool-calling architecture.**

The Needle Agent provides a high-level interface for integrating tool use and schema validation into LLM workflows. As implemented in `cactus-compute/needle`, the agent orchestrates the request-response cycle between Python callables and the native engine, supporting both interactive tool invocation and one-shot structured extraction through three core components defined in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), and [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py).

## Needle Agent Architecture Overview

The agent operates through a three-component system that bridges Python and the underlying C++ engine:

- **`Needle` class** ([`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)): Manages the C library handle, loads optional fine-tuned weights, and orchestrates the request-response cycle. It maintains the `self._functions` registry and implements the `run()` loop that dispatches tool calls.

- **Tool registration** ([`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)): Implements the `@tool` decorator and `build_schema()` function. The decorator attaches a pre-computed JSON schema to callables, while `build_schema()` auto-generates schemas from type hints for undecorated functions.

- **Extraction helper** ([`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), lines 30-55): Provides the `extract()` function for one-shot structured data retrieval, creating temporary agent instances with Pydantic models as the sole tool.

## Tool Calling with the Needle Agent

Tool calling enables the LLM to invoke Python functions during text generation. The implementation stores tool schemas in `self._functions` and executes a multi-step dispatch loop.

### Defining Tools with the `@tool` Decorator

Tools are defined using the `@tool` decorator or raw type hints. The decorator, implemented in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), attaches a JSON schema to the function object:

```python

# needle/agent/tools.py

def tool(fn: Callable) -> Callable:
    fn._needle_tool = build_schema(fn)
    return fn

```

Alternatively, `Needle` auto-generates schemas via `build_schema()` when undecorated functions are passed to the `tools` argument.

### The Tool Execution Loop

The execution flow follows five distinct phases:

1. **Schema resolution**: During initialization, `Needle._resolve` (lines 38-49 in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)) converts each tool into a JSON schema and populates `self._functions`.

2. **Prompt submission**: The `run()` method sends the user query to the engine via `_complete()`.

3. **Function dispatch**: When the engine returns a `"type": "call"` payload, `Needle.run` iterates over `function_calls`, looks up each callable in `self._functions`, and executes it with the provided arguments (lines 84-104):

```python

# needle/__init__.py (simplified from lines 84-104)

for call in calls:
    fn = self._functions.get(call.get("name"))
    results.append(fn(**(call.get("arguments") or {})))
response = self._complete(json.dumps(results, default=_jsonable), max_new_tokens)

```

4. **Result feedback**: Serialized results are fed back to the engine for subsequent reasoning steps, looping up to `max_steps`.

5. **Final aggregation**: The accumulated `executed` list is attached as `response["results"]` and returned to the caller.

## Structured Extraction from Text

Structured extraction leverages the same underlying architecture to parse unstructured text into typed Python objects, typically Pydantic models.

### One-Shot Extraction with Pydantic Models

The high-level `extract()` function (lines 30-55 in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)) creates a temporary agent instance using the supplied schema as the sole available tool:

```python

# needle/__init__.py (lines 30-55)

agent = Needle(tools=[schema], system=system, weights=selected)
response = agent._complete(text, max_new_tokens)
arguments = calls[0].get("arguments") or {}
if strict:
    _validate_extraction(text, schema, arguments, response)
return schema(**arguments) if _is_pydantic_model(schema) else arguments

```

The function parses the first `function_calls` entry to obtain arguments, optionally validates the extraction, and returns either a Pydantic model instance or a plain dictionary.

### Validation and Error Handling

When `strict=True`, the `_validate_extraction()` helper performs temporal grounding checks and negation flag validation. If the extracted arguments contradict temporal constraints or negation markers present in the source text, the function raises `ExtractionValidationError`.

## Setting Up the Native Engine

Before instantiating the `Needle` class, the C++ engine binaries must be fetched from Hugging Face. The `fetch_library()` function in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) (lines 13-31) downloads platform-specific shared libraries (`libneedle.so`, `.dylib`, or `.dll`) and caches them under `~/.cache/cactus-needle`:

```python

# needle/agent/fetch.py (lines 13-31)

path = hf_hub_download(..., filename="python/" + wheel, repo_type="model")
with zipfile.ZipFile(path) as archive:
    data = archive.read("needle/" + lib)

```

This fetch step is mandatory; any `Needle` instantiation will fail if the native library is not present in the cache directory.

## Practical Examples

### Basic Tool Calling Example

This example demonstrates registering a Python function and invoking it through the agent:

```python
from needle import Needle, tool

@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

needle = Needle(tools=[add])
resp = needle.run(
    query="What is the sum of 7 and 12? Use the add tool.",
    max_steps=2,
)
print(resp["results"])

# → [{'result': 19}]

```

The `@tool` decorator attaches the schema metadata in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), while the execution loop resides in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) lines 84-104.

### Structured Data Extraction Example

Extract typed events from unstructured text using Pydantic validation:

```python
from pydantic import BaseModel, Field
from needle import extract

class Event(BaseModel):
    title: str
    date: int = Field(..., description="Year of the event")
    location: str | None = None

text = "The conference called PyCon will happen in 2025 in Boston."
event = extract(text, Event, strict=True)
print(event)

# → title='PyCon' date=2025 location='Boston'

```

This uses the `extract` implementation at lines 30-55 of [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) and the `_validate_extraction` helper for strict mode verification.

### Command-Line Interface Usage

The CLI exposes tool-enabled queries without Python boilerplate:

```bash
needle run \
  --checkpoint my-model.cact \
  --query "Find the latest release version on GitHub." \
  --tools '[{"name":"fetch_github_release","description":"Fetch latest tag","parameters":{"type":"object","properties":{"repo":{"type":"string"}}}}]'

```

The CLI parses the JSON tool definitions in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) (lines 5-30) and constructs the `Needle` instance internally.

## Summary

- The **Needle Agent** wraps the C++ engine to provide Python-native tool calling and structured extraction capabilities.
- Tools are registered via the **`@tool` decorator** or auto-generated schemas from type hints, stored in `self._functions`, and dispatched through the `run()` loop in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).
- **Structured extraction** uses the `extract()` function to create temporary agents with Pydantic models, validating outputs against temporal and negation constraints when `strict=True`.
- The **native engine** is fetched automatically from Hugging Face into `~/.cache/cactus-needle` via [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py).
- Both capabilities share the same underlying architecture, differing only in execution duration (multi-step conversation vs. one-shot completion).

## Frequently Asked Questions

### How does the Needle Agent handle tool schema generation for undecorated functions?

According to the source code in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), undecorated functions passed to the `Needle` constructor are processed by `build_schema()`, which inspects type hints and docstrings to generate JSON schemas dynamically. This allows plain Python functions to be used as tools without requiring the `@tool` decorator, though explicit decoration provides more control over schema metadata.

### What validation occurs when using `strict=True` in structured extraction?

As implemented in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), the `_validate_extraction()` helper checks for temporal grounding consistency and negation flags between the source text and extracted arguments. If the extracted values contradict temporal markers (e.g., past vs. future dates) or improperly handle negated statements in the input, the function raises `ExtractionValidationError` before returning the typed object.

### Can the Needle Agent execute multiple tool calls in a single generation step?

Yes. The `run()` method in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 84-104) iterates over all `function_calls` returned by the engine in a single response, executing each callable sequentially and collecting results. These results are serialized and fed back to the engine in a subsequent completion request, supporting multi-step reasoning workflows up to the specified `max_steps` limit.

### Where does the Needle Agent store downloaded engine binaries and how are they loaded?

The `fetch_library()` function in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) downloads platform-specific wheels from Hugging Face, extracts the shared library (`libneedle.so`, `.dylib`, or `.dll`), and stores it under `~/.cache/cactus-needle`. The `Needle` class in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) loads this library at instantiation time; if the binary is missing, instantiation will fail with an error indicating that `fetch_library()` must be called first.