How Needle 2 Enforces Output Conformity to Declared Schemas

Needle 2 guarantees that model-generated outputs strictly adhere to JSON schemas by normalizing Pydantic models into JSON-Schema dictionaries, generating structured payloads, and recursively validating every field against source text through a three-stage grounding and validation pipeline.

Needle 2 is an open-source structured extraction framework that ensures language model outputs conform precisely to declared schemas. Unlike approaches relying on post-hoc heuristics or gentle prompting, the library implements a rigorous validation pipeline that statically walks JSON schemas and cross-checks concrete values against input text before exposing results to users.

Schema Normalization: Converting Python Types to JSON-Schema

Needle 2 begins enforcement by transforming Python callables and Pydantic models into standards-compliant JSON-Schema dictionaries. In needle/agent/tools.py, the build_schema and pydantic_schema functions handle this conversion, extracting field definitions, descriptions, and format constraints from type annotations.

The resulting schema objects are attached directly to decorated functions via fn._needle_tool or accumulated in the engine's internal _tool_schemas list (lines 173–214 of needle/__init__.py). This attachment ensures that every tool or extraction request carries its structural contract through the execution pipeline.

Generation and Initial Extraction

When processing requests, the public method Needle.extract(text, schema, …) delegates to an internal helper extract defined in needle/__init__.py. This stage leverages needle/model/run.py as a low-level inference wrapper to prompt the underlying language model for a JSON payload matching the requested schema shape. However, Needle 2 treats this initial output as untrusted raw material rather than a validated result.

The generation phase produces structured data that superficially resembles the target schema, but the framework deliberately suspends judgment until the subsequent validation stage completes its verification pass.

Grounding and Validation Pipeline

The core enforcement mechanism occurs in the grounding and validation stage, which recursively verifies that generated values actually exist in or are supported by the source text.

Temporal Source Analysis

Before validating structure, Needle 2 extracts temporal context from the input. The _source_years and _licensed_years methods (lines 78–99 of needle/__init__.py) scan the original input text and optional system prompts to identify all calendar years explicitly mentioned in the source material. These years form the "licensed" temporal bounds that constrain date and date-time fields in the output.

Recursive Schema Walking

The _walk_grounding function implements the primary validation logic. This recursive walker accepts the JSON schema, the generated arguments object, and the set of licensed years, then traverses the data structure according to schema definitions:

• It resolves $ref pointers to handle complex nested schemas.
• For string fields with date or date-time formats, it verifies that the year component appears in the licensed years set extracted from the source.
• For objects and arrays, it follows properties, items, and anyOf/oneOf branches, ignoring nullable branches when encountering null values.
• It records every successfully verified path in a checked collection and logs failures for later analysis.

Final Validation and Error Handling

The _validate_extraction method (lines 68–80 of needle/__init__.py) combines grounding results with any previously recorded ungrounded paths, applies negation checks, and raises an ExtractionValidationError if unresolved failures remain. This exception includes a precise list of schema paths that failed grounding, enabling programmatic retry logic or user-facing error messages that specify exactly which fields violated the schema constraints.

If validation succeeds, the payload returns to the caller with guaranteed structural and semantic conformity to the declared schema.

Implementation Example: Validating Weather Data Extraction

The following demonstration shows schema enforcement in practice, where an invalid date would trigger validation failure:

from needle import Needle, ExtractionValidationError
from pydantic import BaseModel, Field

# Declare schema using Pydantic

class WeatherRequest(BaseModel):
    """Weather query."""
    city: str = Field(..., description="City name")
    date: str = Field(..., format="date", description="Date of interest")

# Initialize engine

engine = Needle()

# Attempt extraction with grounding verification

prompt = "What will the weather be in Paris on 2024-07-15?"
try:
    result = engine.extract(prompt, WeatherRequest)
    print(result)  # → {'city': 'Paris', 'date': '2024-07-15'}

except ExtractionValidationError as err:
    print("Validation failed:", err)

If the model hallucinated "2025-07-15" (a year not present in the prompt), the engine raises ExtractionValidationError because 2025 is not grounded in the source text.

Summary

• Needle 2 enforces output conformity through a three-stage pipeline: schema normalization, structured generation, and recursive grounding validation.
• Schema definitions are built by build_schema and pydantic_schema in needle/agent/tools.py, then attached to tools or stored in _tool_schemas.
• Temporal grounding ensures date and date-time fields reference years actually present in source text via _source_years and _licensed_years.
• Recursive validation via _walk_grounding checks every schema path, resolves references, and tracks failures through checked and failures collections.
• Strict error handling through _validate_extraction raises ExtractionValidationError with specific path information when grounding fails.

Frequently Asked Questions

How does Needle 2 handle complex nested schemas with references?

Needle 2 processes nested structures through the _walk_grounding function, which actively resolves $ref pointers while traversing object properties and array items. The walker handles anyOf and oneOf branches by exploring valid alternatives and ignores nullable branches when encountering null values, ensuring comprehensive validation across arbitrarily deep schema hierarchies defined in needle/__init__.py.

What happens when validation fails during extraction?

When validation fails, Needle 2 raises an ExtractionValidationError that contains a detailed list of schema paths that could not be grounded in the source text. This exception allows calling code to identify exactly which fields violated constraints—such as dates with unsupported years or missing required properties—and implement retry logic with adjusted prompts or fallbacks.

Can Needle 2 validate schemas without temporal grounding?

While Needle 2 specializes in temporal grounding for date and date-time fields, the validation framework is schema-agnostic and checks all field types against their JSON-Schema definitions. The _walk_grounding mechanism verifies structural conformity for objects, arrays, and primitive types regardless of whether temporal constraints apply, though the library is optimized for use cases requiring source-text verification.

Where does schema normalization occur in the codebase?

Schema normalization occurs in needle/agent/tools.py through the build_schema and pydantic_schema functions, which convert Python callables and Pydantic models into JSON-Schema dictionaries. These normalized schemas are then stored in the engine's _tool_schemas list or attached directly to function objects via the _needle_tool attribute, as implemented in needle/__init__.py between lines 173 and 214.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}