How to Use Pydantic Models Directly with the extract() Method in Needle

The extract() function accepts a Pydantic model as its schema parameter and returns a fully validated model instance populated from the LLM output, automatically converting model definitions into tool schemas for structured generation.

The needle library from cactus-compute/needle provides a one-shot API for structured data extraction from unstructured text. When you use Pydantic models directly with the extract() method, you get automatic validation, type coercion, and access to Pydantic's rich ecosystem of validators while the library handles the LLM tool-calling mechanics internally.

How extract() Processes Pydantic Models

According to the source code in [needle/__init__.py](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py#L45-L56), the extract() method treats your Pydantic model as the only available tool for the LLM. It creates a temporary Needle instance with tools=[schema], executes agent.complete() on your input text, and inspects the response for function_calls.

Instantiation Logic (Lines 45-56)

The implementation checks _is_pydantic_model(schema)—a utility defined in [needle/agent/tools.py](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py). If the schema is a Pydantic model, the function extracts the arguments from the LLM's function call and instantiates your model: schema(**arguments). If you pass a plain dictionary schema instead, the raw argument dictionary is returned without validation.

Complete Working Example

This example demonstrates defining a product schema and extracting structured data from free-form text.

Step 1: Define the Pydantic Schema

Define your model with type hints and validation constraints. The Field descriptions help guide the LLM's extraction.

from pydantic import BaseModel, Field
from needle import extract

class ProductInfo(BaseModel):
    name: str = Field(..., description="Name of the product")
    price: float = Field(..., gt=0, description="Price in USD")
    in_stock: bool = Field(..., description="Availability flag")

Step 2: Perform the Extraction

Pass your text and the Pydantic model to extract(). The function returns a fully validated ProductInfo instance.

text = """
The new SmartWidget costs $199.99 and is currently available.
"""

product = extract(text, ProductInfo)

print(product)           # → ProductInfo(name='SmartWidget', price=199.99, in_stock=True)

print(product.dict())    # → {'name': 'SmartWidget', 'price': 199.99, 'in_stock': True}

Dictionary Schemas vs. Pydantic Models

You can also pass a plain dictionary schema compatible with OpenAI's function-calling format. However, this bypasses Pydantic's validation layer and returns a raw dictionary:

schema = {
    "name": {"type": "string"},
    "price": {"type": "number"},
    "in_stock": {"type": "boolean"}
}

result = extract(text, schema)
print(result)  # → {'name': 'SmartWidget', 'price': 199.99, 'in_stock': True}

Key difference: Pydantic models enforce type safety, constraints (like gt=0 for prices), and custom validators at runtime, while dictionary schemas rely entirely on the LLM's output format.

Module-Level vs. Instance Methods

You can invoke extract() as a top-level function or as a method on a Needle instance (exposed in [needle/agent/__init__.py](https://github.com/cactus-compute/needle/blob/main/needle/agent/__init__.py)). Both approaches execute identical logic:

from needle import Needle

needle = Needle()
product = needle.extract(text, ProductInfo)  # Same result as extract(text, ProductInfo)

Summary

• Automatic conversion: Pydantic models passed to extract() are converted to LLM tool schemas via internal utilities in needle/agent/tools.py.
• Validation guaranteed: The function returns instantiated Pydantic model instances (schema(**arguments)) with full validation support, unlike dictionary schemas which return raw arguments.
• One-shot convenience: The implementation in needle/__init__.py handles temporary agent creation, tool binding, and result parsing, eliminating boilerplate for single extractions.
• Flexible invocation: Use either the module-level extract() helper or Needle.extract() on an instance.

Frequently Asked Questions

Does extract() support Pydantic V2 models?

Yes. The _is_pydantic_model() utility in needle/agent/tools.py detects both V1 and V2 BaseModel subclasses, ensuring compatibility with modern Pydantic features like strict mode, model_validator, and field-specific constraints.

What happens if the LLM returns malformed data?

Since extract() instantiates your model using schema(**arguments) (as implemented in lines 45-56 of needle/__init__.py), any validation errors raised by Pydantic—such as type mismatches or constraint violations—propagate directly to your application. This allows you to catch extraction failures explicitly using standard Pydantic exception handling.

Can I use nested Pydantic models for complex extraction?

Yes. The schema is treated as a single tool definition, so nested models work naturally. The LLM generates a JSON object matching the nested structure, and Pydantic handles the recursive instantiation and validation of sub-models automatically when the top-level model is instantiated.

Is there a performance difference between using extract() and a manual Needle agent?

No significant difference exists. The extract() function is a thin convenience wrapper that creates a temporary Needle instance (from needle/agent/__init__.py) with minimal overhead. It exists to reduce boilerplate for one-shot extraction tasks without sacrificing the underlying agent's performance characteristics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}