Understanding the Needle 2 complete() Method Response Contract

The needles.Needle.complete() method returns a single JSON-compatible Python dictionary that represents the engine's response for one conversation turn, guaranteeing specific fields for function calls, error states, and performance metrics.

The complete() method serves as the primary inference interface in the cactus-compute/needle repository. When integrating Needle 2 into agentic workflows, developers must parse the exact shape of this response dictionary to execute tool calls and manage conversation state.

Response Contract Structure

The response contract is defined in needle/__init__.py within the C-extension wrapper implementation (lines 111-126). The method returns a dictionary that is always JSON-serializable, making it safe for network transmission or persistent storage according to the API specification in doc/apis.md (lines 84-96).

Core Operational Fields

Every response dictionary includes the following top-level fields:

• type (str): Indicates the response category. Returns "call" when the model emits a function call, or "respond" when the conversation loop terminates without tool invocations.
• success (bool): True if the engine produced a syntactically valid function call; False if a runtime error occurred during inference.
• error (str | None): Human-readable error message present only when success is False.
• error_code (int | None): Numeric error code from the native library, provided for debugging purposes.
• reasoning (str): Model-generated trace mapping spans of the prompt to extracted arguments (e.g., 'ten minutes' -> minutes 10).

Function Call Objects

When type equals "call", the function_calls field contains a list of zero or more dictionaries, each with:

• name (str): The exact tool name registered with the agent.
• arguments (dict): Key-value pairs of extracted arguments, containing only values evidenced by the prompt context.

Performance Telemetry

The response includes real-time execution metrics for monitoring and optimization:

• prefill_tps (float): Tokens-per-second throughput during the prompt pre-fill stage.
• decode_tps (float): Tokens-per-second throughput during the decoding generation stage.
• peak_ram_mb (float): Peak RAM consumption in megabytes for this specific inference turn.
• confidence (float | None): Calibrated confidence score between 0 and 1 for the generated call; becomes None when using fine-tuned weights.

Implementation Details

According to the source code in needle/__init__.py, the complete() method handles the boundary between Python and the underlying C library. If an error occurs inside the wrapper itself (lines 115-122), the method raises a RuntimeError instead of returning a dictionary. This distinction separates infrastructure failures from model inference errors, which set success=False within the returned dictionary.

Practical Usage Examples

Executing a Single Tool Call

The following pattern demonstrates basic invocation and response handling:

import needle

agent = needle.Needle(tools=[set_lights])   # `set_lights` is a decorated tool

resp = agent.complete("dim the living room to 30")
print(resp["type"])               # → "call"

print(resp["function_calls"])     # → [{'name': 'set_lights', 'arguments': {...}}]

print(resp["confidence"])         # → 0.94 (or None for fine‑tuned weights)

Building Multi-Turn Agent Loops

For conversational agents that require tool execution and follow-up:

import json, needle

agent = needle.Needle(tools=[search, email])
while True:
    # Get model output

    out = agent.complete(user_query)
    # Stop when the model says “respond”

    if out["type"] == "respond":
        break

    # Execute each suggested tool

    results = []
    for call in out["function_calls"]:
        fn = agent._functions[call["name"]]
        results.append(fn(**call["arguments"]))

    # Feed the results back as the next prompt

    user_query = json.dumps(results)

Monitoring Production Performance

Access telemetry data for optimization and debugging:

resp = agent.complete("what's the weather?")
print(f"Decoding speed: {resp['decode_tps']} tps")
print(f"Peak RAM: {resp['peak_ram_mb']} MiB")

Error Handling Behavior

Distinguish between two failure modes when calling complete():

1. Wrapper Runtime Errors: Raised as RuntimeError exceptions when the C-extension binding itself fails (e.g., memory allocation failures in the native layer at lines 115-122).
2. Model Inference Errors: Returned within the dictionary with success=False, providing structured error messages in the error and error_code fields for application-level handling.

Summary

• The complete() method returns a JSON-compatible dictionary with a guaranteed schema for every conversation turn.
• Response type ("call" vs "respond") determines whether the agent should execute tools or terminate the loop.
• Function calls include validated tool names and evidenced arguments extracted from the prompt.
• Performance metrics (prefill TPS, decode TPS, RAM usage) provide operational visibility into inference efficiency.
• Errors manifest either as raised RuntimeError exceptions (wrapper failures) or as structured error fields within the response dictionary (model failures).

Frequently Asked Questions

What is the exact return type of the Needle 2 complete() method?

The method returns a single Python dict that is guaranteed to be JSON-serializable. According to the implementation in needle/__init__.py, the dictionary contains string keys mapping to values of type str, bool, int, float, list, or None, depending on the specific field.

How does the response distinguish between tool calls and final responses?

The type field provides this distinction. When the model generates function calls, type equals "call" and the function_calls list contains the extracted invocations. When the conversation completes without requiring tools, type equals "respond" and function_calls is typically empty.

What happens when the Needle 2 C-extension encounters a critical error?

If the error occurs within the wrapper layer itself (lines 115-122 of needle/__init__.py), the method raises a RuntimeError exception rather than returning a dictionary. This indicates infrastructure-level failures such as native library crashes or memory allocation errors, separate from model inference errors which return success=False.

Are performance metrics available in every complete() response?

Yes. The prefill_tps, decode_tps, and peak_ram_mb fields are always present as floating-point numbers, providing tokenization and memory statistics for every inference turn, regardless of whether the response contains function calls or final text.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}