How to Perform One-Shot Structured Extraction with `agent.extract()` in Cactus Needle

needle.extract() executes a single structured data extraction by instantiating a temporary Needle agent, routing the request through the native completion engine, and returning validated Pydantic instances or raw dictionaries.

The Cactus Needle library provides a streamlined interface for transforming unstructured text into structured data through its top-level extract function. Located in needle/__init__.py, this utility eliminates manual agent lifecycle management by wrapping the native engine—internally referenced as agent.extract() behavior—in a one-shot execution pattern. Whether using Pydantic models for strict type safety or plain dictionaries for flexibility, the function handles initialization, completion routing, and temporal validation automatically.

How the Extraction Pipeline Works

The one-shot workflow defined in needle/__init__.py orchestrates five distinct operations to transform input text into structured outputs.

Engine Generation Selection

When custom weights are supplied via the weights parameter, the function inspects the archive header's _weight_generation field to determine the appropriate engine generation (lines 29-38). If no custom weights are specified, the system defaults to generation 2.

Schema-as-Tool Initialization

The function constructs a temporary Needle agent by passing the extraction schema as a tool parameter (Needle(tools=[schema], …)), optionally incorporating a system prompt to guide the model's behavior (lines 96-100).

Native Completion Execution

The agent invokes the internal _complete method, which returns a JSON-encoded function-call payload. The extraction results reside in the first call's arguments dictionary (lines 500-505).

Temporal Grounding Validation

When strict=True, the function validates that dates and years in the extracted output align temporally with the source text. Mismatches trigger an ExtractionValidationError according to the validation logic in lines 68-80.

Result Instantiation

For Pydantic schemas, the function returns a fully instantiated model instance via schema(**arguments). For dictionary schemas, it returns the raw extracted values (lines 6-8).

One-Shot Extraction Code Examples

The extract function supports multiple schema types without requiring explicit agent management, as demonstrated in the public entry point documented at lines 86-90.

Extracting with Pydantic Models

Using Pydantic models provides runtime type validation and IDE autocompletion support.

from needle import extract, Field
from pydantic import BaseModel

class Contact(BaseModel):
    name: str = Field(..., description="Full name of the person")
    email: str = Field(..., description="Email address")

text = "John Doe can be reached at john@doe.com"
result = extract(text, Contact)          # ← one-shot extraction

print(result)   # → Contact(name='John Doe', email='john@doe.com')

Dictionary Schema Extraction

For dynamic schemas or rapid prototyping, plain dictionary definitions work natively without external dependencies.

schema = {
    "type": "object",
    "properties": {
        "city": {"type": "string"},
        "temperature": {"type": "number"}
    },
    "required": ["city", "temperature"]
}

system = "You are a weather-reporting assistant."
text = "The weather in Paris is 23°C."

payload = extract(text, schema, system=system, strict=False)
print(payload)   # → {'city': 'Paris', 'temperature': 23}

Using Custom Fine-Tuned Weights

Supply custom .cact archives to leverage domain-specific fine-tuning, with automatic generation detection based on the archive header.

result = extract(
    "The invoice is due on 5th September 2024",
    schema,
    weights="my_weights.cact",   # ← selects generation based on header

    strict=True                  # ← raises ExtractionValidationError if year mismatches input

)

Core Source Files and Architecture

Three primary files implement the end-to-end extraction pipeline according to the repository structure:

• needle/__init__.py — Contains the extract function, temporal validation logic, and the Needle class that powers one-shot extraction (lines 86-90 document the public API).
• needle/agent/tools.py — Provides Field, build_schema, and pydantic_schema utilities used to convert Python types into JSON schemas for the extraction tool.
• needle/_worker.py — Houses the FineTuneWorker implementation responsible for loading custom weight archives when the weights parameter is supplied.

Summary

• needle.extract() creates a temporary agent per invocation, eliminating the need for manual lifecycle management in single-use scenarios.
• Engine generation auto-detects from custom weight headers via _weight_generation or defaults to generation 2.
• Schema validation supports both Pydantic BaseModel classes and raw JSON Schema dictionaries through the tools interface.
• Strict mode enforces temporal consistency between input text and extracted dates, raising ExtractionValidationError on mismatch.
• Output formats include fully instantiated Pydantic objects or raw dictionaries depending on the input schema type.

Frequently Asked Questions

What is the difference between extract() and manually creating a Needle agent?

extract() is a stateless convenience wrapper that instantiates a temporary agent for single-use extraction tasks. Manually creating a Needle instance is preferable for batch processing or when maintaining persistent tool states and conversation history across multiple calls.

How does strict=True validate temporal grounding?

According to needle/__init__.py (lines 68-80), strict mode parses dates and years from both the input text and the extracted output, raising ExtractionValidationError if the extracted temporal information contradicts or falls outside the temporal bounds present in the source material.

Can extract() use custom fine-tuned models?

Yes. Pass the path to a .cact archive via the weights parameter. The function automatically reads the _weight_generation header to select the appropriate engine generation, ensuring compatibility with the specific fine-tuned weights.

What schema formats does extract() accept?

The function accepts both Pydantic BaseModel subclasses and JSON Schema dictionaries. Pydantic models are converted internally using utilities from needle/agent/tools.py, while dictionaries are passed directly to the completion engine as structured tool definitions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}