How to Declare Tools Using Raw JSON Schema in Needle

Pass a list of raw JSON schema dictionaries to the Needle constructor's tools parameter to define tools without Python decorators, enabling programmatic tool generation while maintaining full compatibility with the constrained decoding engine.

Needle is a Python library for building LLM agents with constrained decoding capabilities. While the @needle.tool decorator provides a convenient Pythonic interface, you can also declare tools using raw JSON schema in Needle when working with dynamically generated tools, external configuration files, or non-Python tool definitions. This approach sends schema objects directly to the native engine via self._tools_json, bypassing Python reflection while delivering identical structural data.

How Raw JSON Schema Resolution Works

When you instantiate a Needle agent, the constructor processes the tools argument through the internal _resolve method located in needle/__init__.py (lines 38-50). If an entry is a dict, Needle assumes it is already a valid JSON schema and appends it directly to the schema collection without transformation.

elif isinstance(entry, dict):
    schemas.append(entry)

The resolved list is then JSON-encoded and stored in self._tools_json (lines 95-102 of the same file), which passes the schema unchanged to the native engine. The engine builds its constrained decoding grammar directly from this structure, eliminating the overhead of Python introspection while maintaining identical functionality to decorated functions.

Required Schema Format

The native engine expects schemas following the OpenAI function-call format. Each tool must be a JSON object containing:

• name: The function identifier used during generation
• description: Explanation of tool purpose and behavior
• parameters: A JSON Schema object defining argument types, constraints, and required fields

According to the documentation in doc/apis.md, this structure matches exactly what the tool decorator produces internally via build_schema in needle/agent/tools.py. Providing the schema directly bypasses the decorator and reflection step, but the resulting data structure is identical, ensuring the engine receives the same constrained decoding instructions.

Implementation Examples

Inline Raw JSON Schema Definition

Define your tool catalogue directly as Python dictionaries using standard JSON Schema syntax:

import needle

tools = [
    {
        "name": "set_lights",
        "description": "Turn a room's lights on or off and set brightness",
        "parameters": {
            "type": "object",
            "properties": {
                "room": {"type": "string", "description": "which room to control"},
                "on": {"type": "boolean"},
                "brightness": {"type": "integer", "minimum": 0, "maximum": 100},
            },
            "required": ["room", "on"],
        },
    }
]

agent = needle.Needle(tools=tools)
result = agent.run("dim the kitchen to 10")
print(result["results"])

This executes a single inference turn where the model calls set_lights with arguments extracted from the natural language query.

Loading Schemas from External Files

Store tool definitions in JSON files for version control or dynamic updates:

import json
import needle

with open("tools.json", "r", encoding="utf-8") as f:
    tools = json.load(f)

agent = needle.Needle(tools=tools)
agent.run("play jazz in the living room")

The tools.json file should contain a list of schema objects. Needle treats file-loaded dictionaries identically to inline definitions, making this approach ideal for microservice architectures or configuration-driven deployments.

Combining Decorated Functions with Raw JSON

Mix Python-decorated tools and raw schemas in the same agent configuration:

from typing import Literal
import needle

@needle.tool
def set_thermostat(temperature: int, mode: Literal["heat", "cool", "auto"] = "auto"):
    return {"temperature": temperature, "mode": mode}

light_schema = {
    "name": "set_lights",
    "description": "Control lights in a room",
    "parameters": {
        "type": "object",
        "properties": {
            "room": {"type": "string"},
            "on": {"type": "boolean"},
        },
        "required": ["room", "on"],
    },
}

agent = needle.Needle(tools=[set_thermostat, light_schema])
agent.run("make it 22 and turn the kitchen lights on")

The _resolve method processes each entry according to its type, allowing decorated callables and raw dictionaries to coexist seamlessly in the tools list.

Summary

• Raw JSON bypass: Pass dictionaries directly to the tools parameter to skip Python decorator processing while maintaining engine compatibility according to needle/__init__.py.
• OpenAI format: Schemas must include name, description, and parameters following the function-call specification recognized by the constrained decoder.
• Flexible sourcing: Load schemas from external JSON files, define them inline, or mix them with decorated Python functions as processed by the Needle._resolve logic.
• Zero transformation: Dictionary entries are appended directly to the schema list (lines 38-50) and passed unmodified to self._tools_json (lines 95-102).

Frequently Asked Questions

What is the performance impact of using raw JSON schemas versus decorated functions?

There is no runtime performance difference during inference because the native engine receives identical schema structures in both cases. Using raw JSON schemas eliminates the initialization-time Python reflection overhead that occurs when the @needle.tool decorator inspects function signatures via build_schema in needle/agent/tools.py, which may marginally improve startup speed when defining thousands of programmatically generated tools.

Can I validate my raw JSON schemas before passing them to Needle?

Yes. Since the engine expects standard JSON Schema within the OpenAI function-call structure, you can validate your schemas using libraries like jsonschema or Pydantic before instantiation. The build_schema utility in needle/agent/tools.py demonstrates the exact structure the engine requires, which you can reference for validation logic.

Does Needle support external schema registries or dynamic schema loading?

Absolutely. Because Needle._resolve accepts any dictionary matching the expected format, you can fetch schemas from REST APIs, databases, or configuration management systems at runtime. Simply load the JSON response into Python dictionaries and pass the list to the tools parameter. As shown in the file-loading example, Needle imposes no restrictions on the origin of the schema data, only on its structural compliance with the constrained decoding grammar requirements.

How do I debug when a raw JSON schema fails to generate valid tool calls?

Enable detailed logging during agent initialization to inspect self._tools_json after the _resolve method processes your input. Verify that your schema includes all required fields (name, description, parameters) and that the parameters object contains valid JSON Schema with proper type definitions. The engine builds its constrained decoding grammar directly from this structure in needle/__init__.py, so syntax errors in the JSON will propagate as generation failures during the initialization phase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}