Needle `agent.complete()` Response Structure Explained: A Complete Guide

agent.complete() returns a Python dictionary with a type key indicating whether the response is a plain text completion ("final") or a tool invocation request ("call"), plus additional fields like text, function_calls, or confidence depending on context.

This guide breaks down the exact structure of the response object from the Needle inference engine, based on the source code in the cactus-compute/needle repository. Understanding this structure is essential for building reliable applications on top of the Needle agent framework.

Core Response Structure

The Needle.complete() method is implemented in needle/__init__.py as a thin wrapper around the native C++ engine. Internally, it calls the private _complete() method, which:

1. Sends the prompt to the C++ engine via needle_complete
2. Receives a JSON-encoded envelope in a fixed-size buffer
3. Parses the buffer with json.loads
4. Optionally injects a "confidence" key when custom weights are loaded

The result is a flat Python dictionary with the following guaranteed and optional fields.

Guaranteed Fields

Every response contains at least these keys:

Key	Type	Description
type	str	Response category: "final" for text completions or "call" for tool invocations

Conditional Fields Based on Response Type

Depending on type, additional fields appear:

• text (str): The generated text, present only when type == "final"
• function_calls (list[dict]): Tool invocations, present only when type == "call". Each element contains:
  • name — the tool name as defined by the @tool decorator
  • arguments — a dictionary of parameter values

Fields Added by the Wrapper

• confidence (null): Inserted only when a fine-tuned weight file is loaded via the weights parameter. The current engine does not emit calibrated scores, so this is always None

Engine-Passthrough Fields

The C++ engine may include additional metadata such as usage, model, or other diagnostic keys. These are passed through unchanged without validation or transformation.

Code Examples: Working with agent.complete() Responses

Basic Text Completion

The simplest case returns "final" with generated text in the text field:

from needle import Needle

agent = Needle()
resp = agent.complete("Write a short poem about rain.")

print(resp["type"])   # → "final"

print(resp["text"])   # → the generated poem

Handling Tool Invocation Responses

When the model decides to use a registered tool, type becomes "call":

from needle import Needle, tool

@tool
def send_email(to: str, subject: str, body: str):
    return {"status": "sent"}

agent = Needle(tools=[send_email])
resp = agent.complete("Please email the team the summary of today's meeting.")

print(resp["type"])            # → "call"

print(resp["function_calls"])  # → [{'name': 'send_email', 'arguments': {'to': '...', 'subject': '...', 'body': '...'}}]

Processing Function Calls Manually

To execute tool calls and optionally feed results back to the model:

def handle_calls(calls):
    results = []
    for call in calls:
        fn = agent._functions[call["name"]]
        results.append(fn(**call.get("arguments", {})))
    return results

if resp["type"] == "call":
    tool_results = handle_calls(resp["function_calls"])
    # Pass results back for multi-turn tool use

Fine-Tuned Weights Response

Loading custom weights triggers injection of the confidence key:

agent = Needle(weights="my_finetuned.cact")
resp = agent.complete("Summarize the article.")

print(resp.get("confidence"))  # → None

print("confidence" in resp)    # → True (key is explicitly added even though value is null)

Source Code Reference

The response structure is defined and manipulated in these locations:

• needle/__init__.py (lines 24-38): Implements Needle.complete(), parses the JSON envelope from needle_complete, and conditionally adds confidence when self._weights is set
• needle/agent/tools.py: Defines the @tool decorator and schema generation that determines how function_calls entries are structured
• tests/test_inference.py: Unit tests validating type, text, and function_calls presence and types
• tests/test_weights.py: Tests confirming confidence key injection when custom weights are loaded

Summary

• agent.complete() returns a plain Python dictionary, not a custom class
• Always check resp["type"] to branch between "final" (use resp["text"]) and "call" (use resp["function_calls"])
• function_calls is a list of dictionaries with name and arguments keys
• confidence is always None when present; it signals a fine-tuned model is loaded but carries no score data
• Additional engine keys (usage, model, etc.) may appear and should be treated as opaque metadata

Frequently Asked Questions

How do I know if the response contains generated text or a tool call?

Check resp["type"]. If it equals "final", read resp["text"]. If it equals "call", process resp["function_calls"]. The Needle engine never returns both simultaneously.

What is the exact structure of items in function_calls?

Each element is a dictionary with two keys: name (string, the tool's registered name) and arguments (dictionary mapping parameter names to values). This matches the JSON schema generated by the @tool decorator in needle/agent/tools.py.

Why is confidence always None?

The current C++ engine in cactus-compute/needle does not compute calibrated confidence scores for fine-tuned models. The wrapper adds the key anyway for API compatibility and future extension, as implemented in needle/__init__.py lines 34-36.

Can I rely on specific keys beyond type, text, and function_calls?

No. Keys like usage or model are passed through directly from the engine without guarantees. Only the fields documented above are contractually stable; treat engine-specific metadata as diagnostic output subject to change.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}