How to Manage Multi-Turn Conversations with the Needle Agent

The Needle agent orchestrates multi-turn conversations through an iterative execution loop in the run() method, which processes tool calls in cycles until the maximum step limit is reached or the model stops requesting functions.

Managing multi-turn conversations with the Needle agent requires understanding how the cactus-compute/needle repository handles stateful dialogue execution. The SDK maintains conversation flow through a bounded loop that automatically processes tool outputs and feeds results back to the language model. This architecture ensures deterministic behavior while keeping memory usage constant regardless of conversation length.

Architecture of the Multi-Turn Execution Loop

The core multi-turn logic resides in needle/__init__.py, specifically within the Needle.run() method implementation. When invoked, the engine initiates a controlled execution cycle that alternates between model inference and tool execution until termination conditions are met.

Initial Completion Phase

The process begins when run() forwards the user query to the private _complete method. As implemented at line 182 of needle/__init__.py, this method returns a JSON envelope containing the model’s output and any pending function_calls. The presence of these calls determines whether the conversation requires tool execution or can terminate immediately with a direct response.

The Tool Execution Cycle

A for loop iterates up to max_steps times, as defined in lines 190-217 of needle/__init__.py. Each iteration performs four critical operations:

1. Call Detection – Validates that response.get("type") equals "call" and confirms the presence of function_calls in the response (lines 190-193)
2. Argument Validation – Validates extracted calls against registered schemas, rejecting ungrounded arguments when strict mode is enabled (lines 194-202)
3. Function Resolution – Looks up each function in the internal _functions registry, which stores tools registered via decorators, Pydantic models, or raw JSON schemas (lines 202-206)
4. Execution and Capture – Invokes the Python callable, capturing return values or exceptions in an executed list for result propagation (lines 207-213)

Result Propagation

After each execution round, the accumulated results serialize to JSON and return to the model via _complete with ground=False (lines 214-217). This parameter prevents the engine from re-annotating the response, maintaining conversation continuity while allowing the model to observe tool outputs.

Implementing Multi-Turn Conversations

To create a functional multi-turn agent, register tools using the @needle.tool decorator and invoke run() with appropriate step limits. The following example demonstrates processing a compound query requiring multiple tool invocations:

import needle

@needle.tool
def get_weather(city: str):
    """Return a mock weather reading for the given city."""
    return {"city": city, "temp_c": 23, "sky": "sunny"}

agent = needle.Needle(tools=[get_weather])

resp = agent.run(
    "What's the weather in Paris? Then tell me the forecast for tomorrow.",
    max_steps=3,               # allow up to 3 tool-call rounds

    max_new_tokens=256,
)

print("Model reply:", resp["output"])
print("Tool results:", resp["results"])

The results array in the final response contains the serialized return values from each tool execution across all turns.

Configuring Conversation Boundaries

Setting Maximum Steps

The max_steps parameter in run() creates a hard limit on the execution loop. As implemented in the for _ in range(max_steps) construct at line 190 of needle/__init__.py, the loop terminates automatically when reaching this threshold, preventing infinite execution chains even if the model continues emitting tool calls.

Enabling Strict Mode for Tool Validation

By default, Needle operates in strict mode, validating tool arguments against registered schemas. When validation fails, the engine records an error object rather than raising a fatal exception, allowing the conversation to continue with error context available to the model.

@needle.tool
def set_light(room: str, brightness: int):
    """Set a light's brightness."""
    return {"room": room, "brightness": brightness}

agent = needle.Needle(tools=[set_light])

# This fails validation due to missing numeric brightness

response = agent.run("Dim the lights in the living room.")

# Results contain: [{'error': 'ungrounded brightness'}]

Using Pydantic Models for Complex Inputs

You can define tools using Pydantic BaseModel classes for structured argument validation across multiple turns:

from pydantic import BaseModel

class CalculatorInput(BaseModel):
    a: int
    b: int

@needle.tool
def add_numbers(payload: CalculatorInput):
    """Add two integers."""
    return {"result": payload.a + payload.b}

agent = needle.Needle(tools=[add_numbers])
resp = agent.run("Add 7 and 5.", max_steps=2)
print(resp["results"])

# → [{'result': 12}]

Memory Management and State Handling

The Needle SDK implements bounded memory constraints to maintain consistent performance across long conversations. The engine retains a sliding window of 256 tokens for conversation history while pinning only tool schemas as KV caches. This design ensures that memory consumption remains constant regardless of conversation length, as the system does not retain the full conversation context in the model's KV cache.

Error handling occurs gracefully within the execution loop. According to lines 218-221 of needle/__init__.py, unknown tools, validation failures, and exceptions raised by user code are captured as structured error objects within the "results" array, allowing the model to potentially recover or report issues without terminating the session.

Summary

• The run() method orchestrates multi-turn conversations through an iterative loop defined in needle/__init__.py, processing tool calls until max_steps is reached or no further calls are requested
• The _complete method handles both initial inference and result feedback with ground=False to prevent re-annotation of responses during the conversation cycle
• Strict mode validation at lines 194-202 rejects ungrounded arguments, storing structured errors in the results array rather than crashing the execution loop
• Bounded memory architecture uses a 256-token sliding window and schema KV caching to maintain constant memory usage regardless of conversation length
• Final response construction attaches accumulated tool results under the "results" key (lines 218-221) before returning control to the caller

Frequently Asked Questions

How does Needle prevent infinite loops in multi-turn conversations?

The execution loop relies on the max_steps parameter passed to run(), which creates a hard limit on iteration count. As implemented in the for _ in range(max_steps) construct at line 190 of needle/__init__.py, the loop terminates automatically when reaching this threshold, regardless of whether the model continues emitting tool calls.

What happens when a tool receives invalid arguments?

When strict mode is enabled (the default), the validation logic at lines 194-202 checks for grounded arguments. If validation fails, the engine appends an error object describing the issue to the executed results list rather than raising an exception, allowing the conversation to continue with error context available to the model.

How does Needle maintain conversation context across turns?

The SDK maintains a sliding window of 256 tokens for conversation history while keeping tool schemas pinned as KV caches. According to the source implementation, only the tool schemas remain cached between turns, while the active conversation history is windowed to ensure memory usage does not scale with conversation length.

Can I use Pydantic models to define tools for multi-turn conversations?

Yes, the _functions registry accepts tools defined via Pydantic models. You can create a BaseModel class defining your input schema and pass it as the payload parameter to your decorated function. The validation logic at lines 202-206 automatically resolves and validates these schemas during the execution loop.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}