How to Perform Single-Turn Inference with `needle.complete()` in Cactus Needle

needle.complete() executes one inference step against a Needle agent, returning a typed response envelope that contains generated text, tool calls, or refusal signals.

The complete() method is the low-level interface for single-turn inference in the Cactus Needle framework. Unlike the higher-level agent.run() helper that manages multi-turn conversations automatically, complete() gives you explicit control over each exchange. This article explains how the method works, what it returns, and how to use it in practice.

---

Binding the Engine on First Use

When you call complete() for the first time on a fresh Needle instance, the method triggers _bind() internally. This process, located in needle/__init__.py (lines 111-126), performs three critical operations:

• Loads the native C inference engine
• Optionally loads model weights (if specified)
• Registers the tool schema for any tools passed to the Needle constructor

Subsequent calls reuse the bound engine, making repeated inference significantly faster.

---

The Complete Inference Flow

Calling complete() follows a strict execution path that bridges Python and the underlying C library.

Step 1: Forward Request to Native Code

The method passes your text prompt and max_new_tokens limit directly to needle_complete, a C function exposed through the Python bindings.

Step 2: Parse the JSON Response Envelope

The C library writes its response into an internal buffer as JSON. Needle parses this into a Python dict with a consistent structure:

Field	Type	Description
"type"	str	"text", "call", or "refuse"
"function_calls"	list	Tool invocations when type is "call"
"reasoning"	str or None	Chain-of-thought content if enabled
"confidence"	float or None	Confidence score; None when using tuned weights
Performance metrics	various	Token counts, latency measurements

---

Basic Single-Turn Inference Example

Start with the simplest case: a Needle agent with no tools configured.

from needle import Needle

agent = Needle()  # No tools declared

resp = agent.complete("Summarize the plot of 'The Matrix'")

print(resp["type"])           # → "text"

print(resp["function_calls"]) # → [] (empty list, no tool invoked)

print(resp.get("text"))       # The generated summary

Because no tools are registered, the model must respond with direct text generation.

---

Handling Tool Calls in Single-Turn Mode

When you attach tools to the Needle agent, complete() can return function calls that you must execute yourself. This is the defining characteristic of single-turn inference: you manage the conversation loop.

from needle import Needle, tool
import json

@tool
def send_email(to: str, subject: str, body: str):
    """Send an email to a recipient."""
    # In production, integrate with your email service here

    return {"ok": True, "message_id": "msg_12345"}

agent = Needle(tools=[send_email])

# Single turn: model decides to call the tool

resp = agent.complete(
    "email finance@cactus.dev, subject expenses, body coffee $14"
)

print(resp["type"])                      # → "call"

print(resp["function_calls"][0]["name"]) # → "send_email"

print(resp["function_calls"][0]["arguments"])

# → {'to': 'finance@cactus.dev', 'subject': 'expenses', 'body': 'coffee $14'}

To continue the conversation, execute the tool and feed the result back:

# Execute the call yourself

tool_call = resp["function_calls"][0]
result = send_email(**tool_call["arguments"])

# Second complete() call continues the conversation

next_resp = agent.complete(json.dumps(result))
print(next_resp["type"])  # → "text" (model responds to tool result)

This explicit handoff gives you full control over tool execution, logging, error handling, and result transformation.

---

Controlling Generation with max_new_tokens

Limit response length to improve latency or enforce brevity:

agent = Needle()
resp = agent.complete(
    "Explain quantum entanglement in two sentences",
    max_new_tokens=64
)

The max_new_tokens parameter is passed directly to the C engine and affects the inference budget for that single turn only.

---

Complete() vs. Run(): When to Use Which

The Needle framework provides two inference patterns:

• complete() – Low-level, single-turn. You manage tool execution and conversation state. Best for custom orchestration, debugging, or integrating with existing conversation managers.

• run() – High-level, multi-turn. Automatically executes tools and continues conversation until completion. Best for simple use cases where you want a final answer with no intermediate handling.

As documented in doc/apis.md (lines 7-8), complete() is the primitive that run() builds upon.

---

Confidence Scores and Tuned Weights

The response envelope's "confidence" field behaves conditionally based on model configuration. When using tuned weights, confidence is explicitly set to None according to the conditional logic in the source. Standard pretrained models return a float confidence score when available.

---

Summary

• needle.complete() performs single-turn inference against a Needle agent, returning a structured response envelope.
• First call triggers _bind() in needle/__init__.py (lines 111-126) to initialize the native engine and load weights.
• Response always contains "type" ("text", "call", or "refuse") plus optional function_calls, reasoning, and metrics.
• Tool execution is your responsibility — feed results back with another complete() call to continue the conversation.
• Use max_new_tokens to control generation length per inference step.
• Prefer complete() over agent.run() when you need explicit control over the conversation loop.

---

Frequently Asked Questions

What is the difference between complete() and run() in Needle?

complete() executes exactly one inference turn and returns immediately, leaving tool execution and conversation continuation to you. run() wraps complete() in a loop that automatically executes tools and continues until the model returns a final text response. Use complete() for custom orchestration and run() for simple "get me an answer" workflows.

How do I handle multiple tool calls from a single complete() response?

The "function_calls" field is a list. Iterate through all entries, execute each tool in order or in parallel as your application requires, then serialize the results and pass them to the next complete() call. The model expects the results in the same order as the original calls.

Why is confidence sometimes None in the response?

Confidence scores are disabled when using tuned weights. The source code explicitly sets confidence to None in this configuration. When using standard pretrained weights without tuning, confidence scores may be available depending on the model variant.

Can I reuse a Needle agent across multiple complete() calls?

Yes. The engine binding happens once on first use and persists for the lifetime of the Needle instance. This makes repeated inference efficient — create one agent and call complete() multiple times rather than reinstantiating.

Where can I find test examples for complete() usage patterns?

The tests/test_inference.py file in the repository contains comprehensive test cases demonstrating typical complete() usage patterns, including tool invocation handling, response parsing, and edge cases.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}