# What Is needle.extract() Used For? Structured Data Extraction in Python

> Learn how needle.extract() simplifies structured data extraction in Python. Get Pydantic models or dictionaries from free-form text with optional temporal grounding validation. Boost your data processing.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-12

---

**`needle.extract()` is a convenience wrapper that performs one-shot structured extraction from free-form text using the Needle engine, returning instantiated Pydantic models or dictionaries while optionally validating temporal grounding against the source text.**

Needle is an open-source structured extraction engine that uses native C++ backends to parse free-form text into typed schemas. The `needle.extract()` function provides a high-level API for users who need immediate, one-shot extraction without managing a persistent `Needle` agent instance.

## How needle.extract() Works Under the Hood

The implementation lives in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) at lines 83-95, where the function acts as a factory for temporary extraction agents. When invoked, `needle.extract()` performs three core operations:

- **Agent Initialization**: It instantiates a temporary `Needle` agent, passing the supplied schema (either a Pydantic model or JSON-schema dictionary) as a tool list with `tools=[schema]`
- **Engine Invocation**: It calls `agent._complete()`, which forwards the prompt to the native C++ engine via `needle_complete` and parses the JSON envelope returned by the engine
- **Resource Cleanup**: The function ensures `agent.close()` runs in a `finally` block to release native resources regardless of success or failure

```python
def extract(text: str, schema: type | dict, system: str | None = None,
            max_new_tokens: int = 256, weights: str | None = None,
            strict: bool = True) -> object:
    """One-shot structured extraction using the matching native engine."""
    selected = weights
    generation = _weight_generation(selected) if selected else 2
    _track("extract", {"n_tools": 1,
                       "tuned": bool(selected),
                       "generation": generation})
    agent = Needle(tools=[schema], system=system, weights=selected)
    try:
        response = agent._complete(text, max_new_tokens)
    finally:
        agent.close()
    # …validation and return handling omitted for brevity…

```

## Parameters and Configuration Options

The `needle.extract()` signature accepts several parameters that control extraction behavior:

- **`text`**: The input string containing unstructured data to parse
- **`schema`**: Either a Pydantic `BaseModel` subclass or a JSON-schema dictionary defining the output structure
- **`system`**: Optional system prompt string to guide the extraction behavior
- **`max_new_tokens`**: Integer limiting the generation length (default: 256)
- **`weights`**: Path to a custom `.cact` fine-tuned model file; when provided, the function routes to `FineTuneWorker` in [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py)
- **`strict`**: Boolean flag (default: `True`) enabling temporal validation against years mentioned in the source text

## Validation and Strict Mode

When `strict=True` (the default), `needle.extract()` invokes the `_validate_extraction` helper (lines 68-81 in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)) to perform **temporal grounding checks**. This validation ensures that date and year values in the extracted output actually appear in or can be directly inferred from the source text.

If the engine produces values that contradict the input or hallucinate dates not present in the text, the function raises `ExtractionValidationError`. Setting `strict=False` bypasses these checks, returning raw extraction results without temporal verification.

## Practical Code Examples

### Extracting with Pydantic Models

Define your schema as a Pydantic model for type-safe extraction with automatic validation:

```python
import pydantic
from needle import extract

class Contact(pydantic.BaseModel):
    name: str
    email: str

text = "John Doe can be reached at john@doe.com"
contact = extract(text, Contact)
print(contact)  # → Contact(name='John Doe', email='john@doe.com')

```

### Using JSON Schema Dictionaries

For dynamic schemas or runtime-defined structures, pass a plain dictionary conforming to JSON Schema:

```python
schema = {
    "type": "object",
    "properties": {
        "city": {"type": "string"},
        "temp_c": {"type": "number"}
    },
    "required": ["city", "temp_c"]
}

result = extract("Weather in Paris: 21 °C", schema)
print(result)  # → {'city': 'Paris', 'temp_c': 21}

```

### Handling Strict Validation Errors

When strict mode detects ungrounded temporal values, it raises `ExtractionValidationError`:

```python
from needle import extract, ExtractionValidationError

try:
    # Input mentions 2025, but extraction asks for year 2020 → validation error

    extract(
        "Report due on 2025-09-01",
        {"type": "object", "properties": {"year": {"type": "integer"}}}
    )
except ExtractionValidationError as e:
    print("Validation failed:", e)

```

### Using Custom Fine-Tuned Weights

Specify a `.cact` weights file to use custom fine-tuned models instead of the base engine:

```python
extracted = extract(
    "Alice bought 3 apples for $5",
    {
        "type": "object",
        "properties": {
            "buyer": {"type": "string"},
            "quantity": {"type": "integer"},
            "price": {"type": "number"}
        }
    },
    weights="fine_tuned.cact"  # Path to custom weights

)

```

## Summary

- **`needle.extract()`** provides a stateless, one-shot API for structured data extraction without manual `Needle` agent management
- The function resides in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) and delegates to the native C++ engine via `Needle._complete()`
- It accepts both **Pydantic models** and **JSON-schema dictionaries** as extraction targets
- **Strict mode** (enabled by default) validates temporal values against source text, raising `ExtractionValidationError` for ungrounded results
- Support for custom `.cact` weight files enables fine-tuned model inference through the `weights` parameter

## Frequently Asked Questions

### What is the difference between needle.extract() and instantiating a Needle agent directly?

`needle.extract()` is a convenience wrapper that creates a temporary `Needle` agent, runs a single extraction, and automatically cleans up resources. Use it for one-off extractions where you don't need to maintain agent state across multiple calls. For batch processing or persistent configurations, instantiate `Needle` directly with `Needle(tools=[schema])` and reuse the agent instance.

### Can I use needle.extract() without Pydantic installed?

Yes. While Pydantic models provide the best developer experience with type hints and validation, `needle.extract()` accepts standard JSON-schema dictionaries as the `schema` parameter. The engine processes both formats identically, making the function usable in lightweight environments where Pydantic dependencies are undesirable.

### Why does needle.extract() raise ExtractionValidationError for dates that seem correct?

When `strict=True` (the default), the function runs `_validate_extraction` to ensure temporal values like years exist in the source text. If the LLM extracts "2020" but the input only mentions "2025", the validation fails to prevent hallucination. Either verify your input text contains the temporal references you're extracting, or set `strict=False` to disable this grounding check.

### How do I use a custom fine-tuned model with needle.extract()?

Pass the file path to your `.cact` weights file via the `weights` parameter. The function routes to `FineTuneWorker` in [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py) when weights are specified, enabling inference using models fine-tuned for specific extraction domains like medical records or legal documents.