Needle 2 `complete()` Response Structure: Complete Field Reference

Needle.complete() returns a strictly JSON-compatible Python dictionary containing ten top-level keys that encode the model's turn output, including type, function_calls, reasoning, and performance metrics.

This guide breaks down the exact schema returned by the complete() method in the cactus-compute/needle repository. Understanding this contract is essential for parsing tool calls, handling errors, and optimizing inference performance in production agentic workflows.

Anatomy of the complete() Response

The response envelope follows a predictable schema defined in needle/__init__.py and documented in doc/apis.md. Every call returns the following ten fields:

• type (str): The turn's contract. Values are "call" (tool execution), "text" (free-form response), or "refuse" (off-topic rejection).
• success (bool): Indicates whether the model considers the turn successful. Always true for correctly formed calls.
• error (str | null): Human-readable error message if generation failed.
• error_code (int | null): Reserved machine-readable error code (currently unused).
• function_calls (list[dict]): Array of tool invocations. Each entry contains name (the tool identifier) and arguments (a JSON object of inferred parameters).
• reasoning (str | null): Free-form derivation explaining how the model mapped natural language to arguments (e.g., "'ten minutes' → minutes 10").
• confidence (float | null): Calibrated confidence score between 0 and 1. Returns null when using tuned weight files.
• prefill_tps (float | null): Tokens-per-second rate during the prefilling stage.
• decode_tps (float | null): Tokens-per-second rate during the decoding stage.
• peak_ram_mb (float | null): Approximate peak RAM consumption in megabytes during the turn.

The JSON grammar guarantees parseability without errors, making the response safe for direct deserialization in any language.

Parsing function_calls and Tool Execution

The function_calls array drives the agentic loop. When type equals "call", this list contains one or more dictionaries requiring execution.

import json
import needle

@needle.tool
def set_lights(room: str, on: bool, brightness: int = 0):
    """Turn a room's lights on/off."""
    return {"room": room, "on": on, "brightness": brightness}

agent = needle.Needle(tools=[set_lights])
resp = agent.complete("Turn the kitchen lights on at 75%")

# Extract the call details

call = resp["function_calls"][0]
tool_name = call["name"]          # "set_lights"

args = call["arguments"]          # {"room": "kitchen", "on": true, "brightness": 75}

# Execute and feed back

result = set_lights(**args)
next_resp = agent.complete(json.dumps(result))

When the model refuses a request or no tool matches the query, function_calls returns an empty list [] and type shifts to "refuse" or "text".

Performance Metrics and Debugging

The response includes three engine-internal metrics useful for profiling:

• prefill_tps: Measures prompt processing speed (prompt evaluation).
• decode_tps: Measures autoregressive generation speed.
• peak_ram_mb: Tracks memory pressure during the forward pass.

These fields help identify bottlenecks in high-throughput deployments.

Error Handling and Edge Cases

Error states are captured through the error and success fields. When success is false, error contains a descriptive string while error_code remains reserved for future use.

Confidence calibration behaves differently based on model weights. Base models return a calibrated confidence float, but tuned models set this field to null and emit a warning at construction time (as noted in doc/apis.md).

Source Code Implementation

The parsing logic resides in two critical files:

1. needle/__init__.py (lines 11-26): Implements the complete() wrapper that deserializes the native library's JSON payload and conditionally inserts the confidence key.
2. doc/apis.md (lines 82-96): Documents the complete envelope schema with field descriptions and type constraints.
3. tests/test_inference.py: Validates that responses contain mandatory keys (type, function_calls, etc.) and conform to the expected structure.

The contract is strictly enforced by the engine's grammar, ensuring backward compatibility as the API evolves.

Summary

• The complete() response structure in Needle 2 is a flat dictionary with ten standardized keys.
• function_calls contains the primary actionable data, with each entry specifying a tool name and arguments object.
• confidence is null for tuned models but populated for base models.
• Performance fields (prefill_tps, decode_tps, peak_ram_mb) aid in latency and memory optimization.
• The schema is strictly JSON-compatible and defined in needle/__init__.py and doc/apis.md.

Frequently Asked Questions

What are the possible values for the type field in a Needle 2 response?

The type field accepts three string values: "call" when the model emits a tool invocation, "text" for free-form conversational responses, and "refuse" when the input is off-topic or violates safety constraints. This field determines how your application should handle the subsequent function_calls array.

Why is the confidence field null in my complete() responses?

The confidence field returns null when you load a tuned weight file because calibrated confidence scores are only available for base models. According to the source code in needle/__init__.py, the engine emits a one-time warning at construction when tuned weights are detected, indicating that confidence values will not be populated during inference.

How do I handle empty function_calls arrays?

An empty function_calls list indicates the model either refused the request (type: "refuse") or determined no registered tool matched the query (type: "text"). Your application should check the type field before attempting to iterate over function_calls, and fall back to displaying the reasoning field or a default message when no tools are invoked.

Where is the complete() response schema formally defined?

The schema is formally defined in the engine's envelope parser within needle/__init__.py (lines 11-26) and documented in doc/apis.md (lines 82-96). The tests/test_inference.py file provides the enforcement layer, ensuring all responses contain the mandatory keys and type signatures required by the API contract.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}