How Needle's `agent.run()` Loop Manages Multi‑Step Tool Execution

The Needle.run() method implements an iterative agent loop that lets language models invoke Python tools repeatedly, feeding results back into the context until the model decides to stop or reaches the step limit.

The needle package provides a lightweight Python binding around a native C engine for language model inference. Its Needle.run() method in needle/__init__.py orchestrates the core pattern modern agents use: planning, tool invocation, observation, and replanning. This article breaks down exactly how that loop works, using source-accurate details from the cactus-compute/needle repository.

The Agent Loop Architecture

Entry Point: self.complete() and Initial Tool Schema

When you call run(), the method first invokes self.complete() with your original query. This wrapper around needle_complete (defined in needle/model/run.py) sends both the prompt and a JSON schema describing all registered tools to the C engine.

The C engine returns a JSON envelope. When the model wants to use tools, that envelope contains "type": "call" and a "function_calls" array with the requested invocations.

The Iterative Execution Loop

The core loop in Needle.run() runs for a configurable number of steps (default 8). Each iteration follows four distinct phases:

• Extract calls – Parse response.get("function_calls"); break if empty or non‑call type

• Dispatch to Python – Look up each function name in self._functions, invoke with extracted arguments, catch and serialize any exceptions

• Collect results – Append outcomes to per‑step results and global executed list

• Re‑prompt the model – JSON‑encode results (via _jsonable for Pydantic compatibility) and call self.complete() again

This cycle enables multi‑step tool execution where later calls can depend on earlier results.

Tool Dispatch and Error Handling

Function resolution happens against self._functions, populated during Needle.__init__ from the tools parameter. The initialization logic (in needle/__init__.py, with schema building from needle/agent/tools.py) handles:

• Plain Python functions decorated with @tool
• Pydantic models for structured inputs/outputs
• Raw JSON schemas for advanced use cases

If a requested function name is missing, run() injects an error dict into results rather than crashing. Tool exceptions are similarly caught and converted to error payloads, keeping the agent loop robust.

Result Serialization with _jsonable

Before feeding tool outputs back to the C engine, run() uses the helper _jsonable to normalize objects. This ensures Pydantic models, dataclasses, and other Python objects become JSON‑serializable without manual conversion.

Multi‑Step Reasoning in Practice

The loop's power lies in context accumulation. Each complete() call includes the conversation history plus the most recent tool results. The model can:

1. Request a lookup
2. Receive structured data
3. Formulate a follow‑up query based on that data
4. Issue additional tool calls

This matches the ReAct pattern (reasoning + acting) without explicit prompting templates.

Complete Working Example

from needle import Needle, tool, Field

@tool
def search_web(query: str):
    """Fake web search – returns a canned response."""
    return {"answer": f"Result for '{query}'"}

# Initialize agent with tool registry

agent = Needle(tools=[search_web], system="You are a helpful assistant.")

# Execute multi‑step query

response = agent.run(
    "Find the capital of France, then look up its population.",
    max_steps=5,
    max_new_tokens=128,
)

print("Final response:", response)
print("Executed tool calls:", response["results"])

Expected execution flow:

1. First complete() → model calls search_web with "capital of France"
2. Loop processes call, returns {"answer": "Result for 'capital of France'"}
3. Second complete() with result → model calls search_web with "Paris population"
4. Loop ends (no more calls) → final response includes both results in "results" key

The response["results"] array preserves the full execution trace for debugging or logging.

Key Source Files

File	Responsibility
needle/__init__.py	Needle class, run() loop, complete() wrapper, tool initialization
needle/agent/tools.py	@tool decorator, schema extraction, Pydantic integration
needle/model/run.py	Low‑level needle_complete C bindings, response parsing

Configuration Parameters

• max_steps – Hard limit on tool invocation rounds (default 8)
• max_new_tokens – Generation budget for each complete() call
• system – System prompt set at initialization

These parameters balance latency, cost, and completion quality for your specific use case.

Summary

• Needle.run() implements a generic agent loop in needle/__init__.py with configurable iteration limits
• Each cycle invokes self.complete() to get tool requests, executes Python functions from self._functions, and feeds JSON results back via _jsonable serialization
• The loop enables multi‑step tool execution where model reasoning can build on prior observations
• Error handling is defensive: unknown functions and tool exceptions become structured error payloads rather than loop‑terminating failures

Frequently Asked Questions

What happens if a tool raises an exception?

The run() loop catches the exception, converts it to an error dict with the exception message, and includes that in the results fed back to the model. The loop continues, allowing the model to recover or report the failure.

How does Needle know which Python functions to call?

During __init__, the agent resolves the tools parameter—functions, Pydantic models, or raw schemas—and populates self._functions with name-to-callable mappings. Tool names in model requests must match these registered keys exactly.

Can the model make multiple tool calls in a single step?

Yes. The "function_calls" array can contain multiple requests. run() iterates through all of them, executes each, and aggregates results before the next complete() call.

Why is there a default limit of 8 steps?

The max_steps default prevents infinite loops from runaway agents. You can increase it for complex workflows requiring extended reasoning chains, or decrease it for latency‑sensitive applications.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}