How Byte-Level Grammar Constraints Ensure Valid Tool Calls in Needle 2

Needle 2 validates every tool call by enforcing a deterministic finite automaton over UTF-8 bytes, ensuring the language model's output conforms to the tool's JSON schema before execution occurs.

Needle 2 introduces a rigorous validation mechanism that prevents malformed or malicious tool invocations at the source. The byte-level grammar constraint transforms each Python function decorated with @tool into a precise JSON schema, then compiles that schema into a byte-level grammar that validates every character of the model's output. This approach guarantees that only syntactically correct, type-safe payloads reach your functions, as implemented in the cactus-compute/needle repository.

Architecture of the Validation Pipeline

The byte-level grammar constraint operates through a five-stage pipeline that bridges Python function definitions and deterministic output validation.

Tool Definition via the @tool Decorator

The process begins when a developer marks a function as a tool using the @tool decorator defined in needle/agent/tools.py. This decorator introspects the function at import time and stores a JSON schema representation on the function object itself.

# Simplified from needle/agent/tools.py lines 68-71

def tool(fn):
    fn._needle_tool = build_schema(fn)
    return fn

The decorator attaches the schema to fn._needle_tool, making the contract metadata available to the inference runtime without requiring runtime introspection overhead.

Schema Construction with build_schema

The build_schema function performs static analysis of the decorated callable to generate a JSON Schema-compliant definition. According to needle/agent/tools.py, this involves several precise steps:

• Signature Introspection: Uses inspect.signature to enumerate parameters and typing.get_type_hints to resolve forward references and complex types.
• Type Mapping: Converts Python types to JSON Schema types via the _JSON_TYPES mapping (lines 8-10).
• Complex Type Handling: Recursively processes enums, Literal types, lists, dictionaries, unions, and nested Pydantic models (lines 64-84).
• Constraint Extraction: Applies Field metadata such as regex patterns, minimum/maximum values, and enumerations to the schema (lines 36-44).

This static analysis ensures that the schema accurately reflects the Python function's type hints and validation constraints at the byte level.

Byte-Level Grammar Generation

Once the JSON schema is constructed, Needle 2 compiles it into a byte-level grammar—specifically, a deterministic finite automaton (DFA) that operates over UTF-8 byte sequences. This grammar precisely describes every valid byte sequence that can be parsed into a JSON object matching the schema.

Unlike higher-level validation that parses JSON then checks schema compliance, the byte-level grammar validates the raw output stream character-by-character. This means deviations such as malformed quotes, missing commas, or incorrect escape sequences are caught at the exact byte position where they occur.

Model Prompting and Grammar Enforcement

During inference, the LLM receives the generated byte-level grammar alongside the tool description. The model is constrained to generate output that strictly follows the DFA's transition rules. This enforcement happens at the sampling layer, where the grammar masks invalid next tokens, ensuring that the model cannot produce syntactically invalid JSON structures or type violations (such as passing an integer where a string is required).

Runtime Validation in fetch.py

The needle/agent/fetch.py module handles the final validation stage. When the model returns a byte stream:

1. Grammar Parsing: The byte stream is fed into the DFA parser.
2. Acceptance Check: If the stream reaches an accepting state, it is safely parsed into a Python dictionary.
3. Rejection and Retry: If the stream fails to match the grammar (e.g., missing required fields or type mismatches), the call is discarded and the model is prompted to regenerate the payload.

This guarantees that only syntactically and semantically valid tool calls ever reach the host environment.

Why Byte-Level Validation Matters

The byte-level grammar constraint provides three critical advantages over traditional post-hoc JSON validation:

• Safety: Prevents malformed or malicious payloads from reaching the host environment by rejecting invalid strings before they are parsed into objects.
• Determinism: Guarantees that the LLM's tool output is reproducible and strictly adheres to the contract expressed by the developer's type hints and Field constraints.
• Developer Experience: By declaring constraints directly in the function signature (e.g., Field(..., pattern="^\\d+$")), developers receive automatic, byte-accurate validation without writing custom parsers or error-handling logic.

Implementation Example

The following example demonstrates how the schema generation and byte-level constraint work together in practice:

from needle.agent.tools import tool, Field

@tool
def translate(
    text: str,
    target_lang: str = Field(enum=["es", "fr", "de"])
) -> str:
    """Translate text into the specified language."""
    return f"Translated '{text}' to {target_lang}"

# The decorator generates this JSON schema internally:

# {

#   "name": "translate",

#   "parameters": {

#     "type": "object",

#     "properties": {

#       "text": {"type": "string"},

#       "target_lang": {"type": "string", "enum": ["es", "fr", "de"]}

#     },

#     "required": ["text", "target_lang"]

#   }

# }

# Valid byte sequence accepted by the grammar:

# {"text":"Hello world","target_lang":"es"}

# Invalid sequences rejected at byte-level (type mismatch):

# {"text":"Hello world","target_lang":123}

# Invalid sequences rejected (missing required field):

# {"target_lang":"es"}

If the model attempts to emit "target_lang": 123 or omits the required text field, the byte-level parser in needle/agent/fetch.py rejects the output before the translate function is ever invoked, triggering a retry with the grammar constraint still enforced.

Summary

• Static Schema Generation: The @tool decorator in needle/agent/tools.py uses build_schema to convert Python type hints and Field constraints into JSON schemas at import time.
• Byte-Level Grammar: Schemas are compiled into deterministic finite automata that validate UTF-8 byte sequences character-by-character, not just parsed objects.
• Runtime Enforcement: needle/agent/fetch.py parses model output against the grammar, rejecting invalid calls and prompting retries without exposing the host to malformed data.
• Type Safety: Supports complex Python types including enums, Literals, unions, and Pydantic models, mapping them precisely to JSON Schema constraints.
• Architectural Safety: The constraint prevents both syntax errors (malformed JSON) and semantic errors (type mismatches, constraint violations) at the generation layer.

Frequently Asked Questions

What is a byte-level grammar constraint?

A byte-level grammar constraint is a validation mechanism that treats the language model's output as a stream of UTF-8 bytes and validates it against a deterministic finite automaton derived from a JSON schema. Unlike traditional validation that parses JSON then checks validity, this approach rejects invalid characters as they are generated, ensuring only byte sequences that conform exactly to the schema reach the parser.

How does Needle 2 handle invalid tool call outputs?

When the runtime validator in needle/agent/fetch.py detects a byte sequence that violates the grammar—whether due to syntax errors like missing braces or semantic errors like passing an integer to a string field—it discards the invalid output and requests a regeneration from the model. The grammar constraint remains active during the retry, ensuring subsequent attempts must also conform to the valid byte patterns defined by the tool's schema.

Which Python types are supported in tool schema generation?

The build_schema function in needle/agent/tools.py supports primitive types mapped via _JSON_TYPES, as well as complex constructs including Enum, Literal, List, Dict, Union types, and nested Pydantic models. The system also respects Field metadata such as pattern, min_length, max_length, and enum constraints, translating them into corresponding JSON Schema validation keywords.

Where does the runtime validation occur in the Needle codebase?

Runtime validation occurs primarily in needle/agent/fetch.py, which receives the raw byte stream from the language model and processes it through the byte-level grammar parser. The validation logic uses the schema attached by the @tool decorator (stored in fn._needle_tool) to instantiate the correct deterministic finite automaton for each specific tool call being validated.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}