How to Configure Needle for Multi‑Turn Agent Loops: A Complete Guide

Configure Needle's multi‑turn agent loops by setting max_steps and max_new_tokens in the run() method, supplying tools via the tools parameter, and optionally customizing the system prompt for persistent behavior across iterations.

Needle is a lightweight, open‑source inference engine from cactus‑compute/needle that supports agentic workflows—conversations where the model repeatedly calls tools, observes results, and decides on subsequent actions. This article explains how to configure and control these multi‑turn loops using the Needle.run() method and its parameters.

Understanding the Multi‑Turn Loop Architecture

The core mechanism lives in needle/__init__.py. When you invoke run(), Needle executes the following sequence:

1. Initial completion — generates a response to your query.
2. Function call detection — checks response.get("function_calls") and response.get("type") == "call".
3. Tool execution — resolves each call against self._functions, executes the matching function, and collects results.
4. Result feedback — serializes results to JSON and feeds them back to the model.
5. Iteration — repeats until no more calls are requested or max_steps is reached.

Here is the actual implementation from the source:

def run(self, query, max_steps=8, max_new_tokens=256):
    response = self.complete(query, max_new_tokens)
    executed = []
    for _ in range(max_steps):
        calls = response.get("function_calls") or []
        if response.get("type") != "call" or not calls:
            break
        results = []
        for call in calls:
            fn = self._functions.get(call.get("name"))
            # ... validation and execution ...

            results.append(fn(**(call.get("arguments") or {})))
        executed.extend(results)
        response = self.complete(json.dumps(results, default=_jsonable), max_new_tokens)
    response["results"] = executed
    return response

The loop requires no manual state management—the engine handles re‑binding and context continuation automatically via self._bind() inside complete.

Configuring Loop Behavior: Key Parameters

Control multi‑turn depth and response characteristics through these run() parameters:

Parameter	Purpose	Default	When to Adjust
max_steps	Hard limit on tool‑calling rounds	8	Increase for complex reasoning chains; decrease to limit API costs
max_new_tokens	Token budget per generation	256	Raise for verbose explanations or structured outputs
system	Persistent system prompt	""	Set personality, constraints, or output formats
tools	Callable capabilities	None	Define what functions the agent can invoke

These parameters apply per run() invocation, allowing dynamic adjustment without re‑instantiating the agent.

Practical Configuration Examples

Basic Multi‑Turn Setup

Create an agent with a single tool and run a conversation requiring follow‑up reasoning:

from needle import Needle, tool

@tool
def get_weather(city: str):
    """Retrieve weather information for planning decisions."""
    return {"temp_c": 22, "condition": "sunny"}

agent = Needle(tools=[get_weather])

result = agent.run(
    "What's the weather in Lisbon? Should I pack a jacket?",
    max_steps=3  # sufficient for query → tool call → final answer

)

print(result["type"])      # "text" when loop terminates

print(result["results"])   # [{"temp_c": 22, "condition": "sunny"}]

This pattern—demonstrated in tests/test_inference.py lines 33–45—shows the minimal configuration for functional agent loops.

Extended Reasoning with Custom Limits

For tasks requiring multiple sequential tool calls, increase both step and token budgets:

@tool
def search_flights(origin: str, destination: str, date: str):
    """Find available flights."""
    return [{"flight": "AB123", "price": 299}]

@tool
def book_flight(flight_id: str):
    """Confirm a reservation."""
    return {"status": "confirmed", "booking_ref": "XYZ789"}

travel_agent = Needle(
    tools=[search_flights, book_flight],
    system="You are a meticulous travel assistant. Always confirm prices before booking."
)

itinerary = travel_agent.run(
    "Find me a flight from NYC to London on March 15, then book the cheapest option.",
    max_steps=10,           # allow search → comparison → booking sequence

    max_new_tokens=512      # longer responses for fare explanations

)

The system prompt persists across all turns, ensuring consistent behavior without repetition.

Controlling Loop Termination

The loop exits automatically when either condition becomes true:

• response.get("type") != "call" — the model returns a final text response instead of tool requests.
• max_steps exhausted — safety limit reached, returning accumulated results.

This design prevents infinite loops while preserving partial progress. Check result["type"] to determine exit reason:

result = agent.run("Complex multi-step query", max_steps=5)

if result["type"] == "call":
    print("Warning: step limit reached before completion")
    # result["results"] contains intermediate tool outputs

Tool Definition and Registration

Tools must be registered for the loop to resolve function calls. Needle supports two patterns, both processed by needle/agent/tools.py:

Decorator pattern (recommended):

from needle import tool

@tool
def calculate_distance(lat1: float, lon1: float, lat2: float, lon2: float):
    """Calculate great-circle distance between coordinates."""
    # implementation...

    return {"km": distance}

Pydantic model pattern — for complex validation, though the decorator handles most use cases.

Tools are bound to self._functions during Needle initialization and remain available for unlimited invocations across all turns.

Performance and Resource Considerations

• Each turn incurs a full inference call — factor max_steps into latency and cost estimates.
• Token accumulation — conversation history grows with each iteration; monitor context window limits in underlying engines.
• max_steps as circuit breaker — set conservatively for production deployments to prevent runaway loops.

The needle/agent/fetch.py module handles engine library loading; customize paths only if running non‑standard deployments.

Summary

• Needle.run() implements the multi‑turn loop in needle/__init__.py with automatic tool resolution and result feedback.
• Control depth via max_steps (default 8) and response length via max_new_tokens (default 256).
• Supply capabilities through tools and persistent behavior through system.
• The loop terminates naturally when the model stops calling tools or when safety limits trigger.
• No additional configuration is required—Needle handles state management and engine binding internally.

Frequently Asked Questions

What happens if the model keeps calling tools indefinitely?

Needle enforces the max_steps parameter as a hard ceiling. Once reached, the loop exits and returns accumulated results in response["results"], with response["type"] still "call" if more calls were pending. Increase max_steps for deeper reasoning, or add logic in tools to signal completion.

Can I modify the system prompt between turns?

The system parameter is set at Needle initialization and persists across all turns in a single run() call. For different behavior in subsequent calls, create a new agent instance or implement turn‑specific instructions within your tools' return values.

How do I add custom validation or logging to the loop?

The run() method does not expose hooks, but you can subclass Needle and override run() or complete() to inject middleware. The source structure in needle/__init__.py makes this straightforward—the loop logic is self‑contained and tool resolution occurs through self._functions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}