# How to Perform Structured Extraction Using Needle's Standard Tool‑Calling Interface

> Learn how to perform structured extraction with Needle's standard tool calling interface. Leverage Pydantic models for type-safe, validated byte-level output.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Needle treats structured extraction as a special case of tool calling, converting Pydantic models or dictionary schemas into single‑tool grammars that enforce type‑safe, validated output at the byte level.**

Structured extraction transforms unstructured text into typed data objects. In the `cactus-compute/needle` repository, this capability is built directly on the library's tool‑calling infrastructure rather than as a separate subsystem. The `extract()` function leverages the same grammar‑constrained generation that powers multi‑tool agents, but restricts the engine to a single extraction schema.

## How Structured Extraction Works Under the Hood

The extraction pipeline in Needle follows four distinct phases, each implemented in specific source modules.

### Schema Generation from Pydantic or Dict

Needle accepts two schema formats for extraction targets. For **Pydantic models**, `pydantic_schema()` in [[`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py#L49-L60) generates a JSON‑Schema compatible description. For **plain callables**, the `@tool` decorator applies `build_schema()` (lines 10‑41 in the same file) to construct the schema from function signatures and docstrings.

### Engine Re‑initialization with Single‑Tool Grammar

The `extract()` implementation in [[`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py#L45-L56) creates a temporary `Needle` instance configured with exactly one tool: the generated extraction schema. This triggers the underlying engine to load a grammar that accepts only the declared fields, rejecting any malformed output at generation time.

### Model Completion via Native Engine

Text is fed through `agent.complete()`, which returns a JSON payload containing `function_calls` with the predicted arguments mapped to the schema fields.

### Result Construction and Type Enforcement

The first (and only) function call's arguments are unpacked. For Pydantic schemas, Needle instantiates the model class; for dict schemas, it returns the raw arguments dictionary (lines 54‑55 of the `extract` function). The byte‑level grammar guarantees type safety—malformed fields cannot be generated.

## Basic Extraction Patterns

### Extract with a Pydantic Model

The most common pattern uses a Pydantic `BaseModel` to define your extraction target:

```python
from needle import extract
from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = extract(text, Invoice)          # → Invoice instance

print(invoice.vendor, invoice.total)       # → Acme Corp 1200.0

```

### Extract with a Plain Dictionary Schema

For dynamic schemas or integration with external systems, pass a dictionary following the JSON‑Schema structure:

```python
from needle import extract

schema = {
    "name": "contact",
    "parameters": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "email": {"type": "string", "format": "email"}
        },
        "required": ["name", "email"]
    }
}

text = "John Doe can be reached at john@doe.com"
contact = extract(text, schema)            # → dict

print(contact)                             # → {'name': 'John Doe', 'email': 'john@doe.com'}

```

## Advanced Extraction Techniques

### Define Custom Extraction Tools with `@tool`

The `@tool` decorator lets you define extraction schemas as annotated Python functions. The function body is never executed—only the signature and docstring matter for schema generation:

```python
from needle import tool, Needle

@tool
def parse_contact(text: str):
    """Extract a name and email from a free‑form sentence."""
    pass   # schema is generated from signature and docstring

agent = Needle(tools=[parse_contact])
result = agent.run("Extract John Doe's email from: john@doe.com")
print(result["results"])   # → [{'name': 'John Doe', 'email': 'john@doe.com'}]

```

### Combine Extraction with Agentic Workflows

Extraction integrates cleanly with multi‑step agent loops. Extract structured data first, then pass results to other tools:

```python
from needle import Needle, extract, tool
import pydantic

@tool
def get_weather(city: str):
    """Get the current weather for a city."""
    return {"temp_c": 22}

class Contact(pydantic.BaseModel):
    name: str
    email: str

agent = Needle(tools=[get_weather])
contact = extract("Jane Smith – jane@smith.io", Contact)
weather = agent.run(f"What's the weather in London?")
print(contact, weather["results"])

```

## Key Source Files for Structured Extraction

| File | Purpose | Lines of Interest |
|------|---------|-----------------|
| [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) | Public API and `extract()` implementation | 45–56 |
| [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) | Schema building (`build_schema`, `pydantic_schema`) and `@tool` decorator | 10–41, 49–60 |
| [`tests/test_inference.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_inference.py) | Verified extraction test cases | Full suite |
| [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) | Feature overview and quick‑start examples | Extraction section |

## Summary

- **Structured extraction in Needle** uses the standard tool‑calling interface with a single‑tool constraint.
- **Schema sources**: Pydantic models (via `pydantic_schema`) or dict schemas (via `build_schema`).
- **Type safety** is enforced at the generation level through grammar constraints, not post‑hoc validation.
- **Integration**: `extract()` creates temporary `Needle` instances; `@tool` enables reusable extraction definitions.
- **Output formats**: Pydantic model instances for typed schemas, dictionaries for dynamic schemas.

## Frequently Asked Questions

### What is the difference between `extract()` and using `@tool` directly?

`extract()` is a convenience wrapper that builds a single‑tool schema and runs extraction in one call. Using `@tool` explicitly gives you reusable tool definitions that can be composed into larger agent workflows. Both use identical schema generation machinery in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py).

### Does Needle validate extracted data against the schema?

Validation occurs at the **generation level** through constrained decoding grammars, not as a post‑processing step. The model physically cannot produce tokens that violate the schema. For Pydantic models, the extracted arguments are also passed through Pydantic's constructor, which applies standard validation rules.

### Can I extract nested or complex data structures?

Yes. The `pydantic_schema()` helper in [`tools.py`](https://github.com/cactus-compute/needle/blob/main/tools.py) handles nested Pydantic models, lists, and standard JSON‑Schema constructs. The grammar engine supports recursive structures as long as they can be expressed in JSON‑Schema format.

### How does structured extraction compare to multi‑tool agent calls?

**Structured extraction** restricts the engine to exactly one tool, guaranteeing the output matches your schema. **Multi‑tool agents** allow the model to select from multiple tools and potentially make multiple calls. Internally, both use `agent.complete()` and the same `function_calls` response format—the difference is tool list cardinality and result handling logic.