Needle 2 Tool Calls JSON Output Format: Schema and Implementation Details

Needle 2 returns LLM responses as a JSON object containing model (integer ID), resets (integer counter), and function_calls (array), where each array element specifies a tool name and an arguments object mapping parameter names to values.

Needle 2 is an open-source inference engine that standardizes tool-calling outputs into a strict JSON schema. When the LLM decides to invoke tools, the response is serialized into a predictable format that client applications can parse reliably. This guide documents the expected JSON output format for tool calls from Needle 2, referencing the actual implementation in the cactus-compute/needle repository.

JSON Output Schema Structure

The complete() method in needle/__init__.py returns a JSON-encoded string with a consistent top-level structure.

Top-Level Response Object

Every response contains three mandatory fields:

• model: Integer identifier of the LLM that generated the response (e.g., 11 or 29).
• resets: Integer count of internal state resets triggered during the request.
• function_calls: Array of tool invocations. If no tools are called, this is an empty array [].

Function Call Object

Each element in the function_calls array is an object with:

• name: String matching the tool name defined in the tool schema.
• arguments: JSON object mapping parameter names to provided values. Optional parameters may be omitted.

{
  "model": 11,
  "resets": 0,
  "function_calls": [
    {
      "name": "add",
      "arguments": { "x": 3, "y": 5 }
    }
  ]
}

Parsing Tool Call Responses in Python

When calling the engine programmatically, the raw result requires JSON parsing.

from needle import Needle
import json

# Define tool schema

tools = [{"name": "add", "parameters": {"type": "object", "properties": {
    "x": {"type": "number"}, "y": {"type": "number"}
}}}]

engine = Needle(tools=tools)
raw_result = engine.complete("What is 3+5?", max_new_tokens=8)
result = json.loads(raw_result)

print(result["model"])          # 11

print(result["function_calls"][0]["name"])      # "add"

print(result["function_calls"][0]["arguments"]) # {"x": 3, "y": 5}

HTTP API Output Format

When using the playground HTTP server, the response body follows the identical schema.

curl -X POST http://localhost:8000/complete \
     -H "Content-Type: application/json" \
     -d '{"query":"What is 2+2?"}'

The response body contains the JSON string with model, resets, and function_calls.

Implementation in the Needle 2 Codebase

The JSON structure is enforced by specific components:

needle/_worker.py: The low-level worker assembles the final JSON payload, populating the model, resets, and function_calls fields before returning the serialized string.

needle/__init__.py: The Needle class orchestrates prompt construction and calls the worker. The complete() method returns the raw JSON string generated by the worker.

needle/model/tokenizer.py: Defines special token markers (e.g., <tool_call> and </tool_call>) used to delimit the JSON payload within the LLM prompt context.

Test Validation:

• tests/test_worker.py: Asserts the presence of model and resets fields in the output.
• tests/test_inference.py: Validates that function_calls entries contain the correct name field.
• tests/test_render.py: Verifies that the rendered prompt includes properly formatted <tool_call> JSON blocks.

Summary

• Needle 2 returns tool call results as a JSON object with three mandatory fields: model, resets, and function_calls.
• Each tool invocation specifies a name string and an arguments object containing parameter mappings.
• The response format is identical whether using the Python complete() method or the HTTP API endpoint.
• Internal implementation in needle/_worker.py ensures consistent schema enforcement, validated by the test suite.

Frequently Asked Questions

What fields are always present in the Needle 2 JSON response?

Every response includes model (integer), resets (integer), and function_calls (array). These fields are populated by the worker in needle/_worker.py and validated in tests/test_worker.py.

How are tool arguments structured in the JSON output?

Tool arguments appear as a JSON object assigned to the arguments key within each function call object. This object maps parameter names to their values, with optional parameters omitted if not provided by the model.

What does the JSON output look like when no tools are called?

When the LLM does not invoke any tools, the function_calls field returns an empty array [], while model and resets still contain their respective integer values.

Which source files control the JSON output format in Needle 2?

The format is generated in needle/_worker.py, exposed through needle/__init__.py via the complete() method, and delimited by tokens defined in needle/model/tokenizer.py. Test coverage ensuring schema compliance exists in tests/test_worker.py, tests/test_inference.py, and tests/test_render.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}