How Needle Handles Multiple Tool Selection in a Single Turn: Architecture and Implementation

Needle enables single-turn multi-tool execution by parsing declarative tool-call markers, dispatching async operations through a centralized queue, and reintegrating results back into the LLM response.

Needle, an open-source agent framework from cactus-compute/needle, allows language models to invoke several tools within one interaction. This architecture eliminates the need for multiple round-trips between the user and the model when solving complex, multi-step tasks. Understanding how Needle handles multiple tool selection single turn reveals a sophisticated three-stage pipeline designed for concurrency and reliability.

The Single-Turn Multi-Tool Pipeline Architecture

Needle’s agent architecture processes multiple tool selections through a deterministic pipeline implemented across needle/agent/tools.py and needle/agent/fetch.py. The workflow divides responsibilities into distinct phases: extraction, dispatch, and integration.

Stage 1: Tool-Call Extraction

After the LLM generates a response, the system scans the text for tool-call markers using a regex-based parser. The extract_tool_calls() function in needle/agent/tools.py identifies blocks formatted as <<tool:NAME>>...<<end>>, collecting all invocations in the order they appear. This declarative syntax, defined in the prompts within needle/cli.py, ensures deterministic parsing regardless of tool complexity.

Stage 2: Batch Dispatch via Async Queue

Extracted calls move to needle/agent/fetch.py, where the run_tool_queue() function creates a dispatch queue. Each entry contains the tool name, arguments, and a reference to the original LLM output. The system processes these entries sequentially within an asyncio event loop, executing each tool in an isolated coroutine. This design maintains low round-trip latency while preventing blocking operations from stalling the main thread.

Stage 3: Result Integration with ToolResultRewriter

As tools complete, the ToolResultRewriter class (also in needle/agent/tools.py) injects outputs back into the original response. The rewriter replaces placeholder blocks with concrete results—whether fetched HTML, generated images, or computed data—and appends metadata comments (<!-- tool:NAME result -->) to preserve context for downstream reasoning. The final enriched response returns to the user in a single turn.

Key Design Patterns for Multi-Tool Reliability

Several architectural decisions ensure robust handling when multiple tools execute simultaneously.

Atomic Queue Processing

Each tool call processes atomically; failures convert to error placeholders rather than crashing the pipeline. This isolation guarantees that one failed tool does not abort subsequent operations in the queue.

Extensible TOOLS_REGISTRY

Tools register in a global dictionary called TOOLS_REGISTRY within needle/agent/tools.py. Adding new capabilities requires only defining a callable matching the expected signature and registering it in this dictionary, making the system modular and plugin-friendly.

Concurrency Safety

All tool invocations run inside an asyncio event loop, allowing the agent to scale to dozens of concurrent calls without blocking the main execution thread.

Implementing Multiple Tool Selection in a Single Turn

Needle’s single-turn capability shines when handling complex requests like fetching content, processing it, and rendering the output. Consider how the system handles a prompt requesting data retrieval, summarization, and markdown formatting:

# Example prompt structure generated for the LLM

prompt = """
Please fetch the latest blog post from https://example.com/blog, 
summarize its content, and render the summary as a markdown table.

<<tool:fetch>>
url=https://example.com/blog
<<end>>

<<tool:summarize>>
text={{fetch_result}}
<<end>>

<<tool:render>>
format=markdown
content={{summarize_result}}
<<end>>
"""

When processing this output, the agent executes the following workflow:

from needle.agent.tools import extract_tool_calls, ToolResultRewriter
from needle.agent.fetch import run_tool_queue

# 1. Parse all tool blocks from the LLM output

calls = extract_tool_calls(prompt)

# 2. Execute tools asynchronously in order

results = await run_tool_queue(calls)

# 3. Stitch results back into the original text

final_output = ToolResultRewriter(prompt, results).rewrite()
print(final_output)

The resulting final_output contains the fetched HTML, the processed summary, and the rendered markdown table—all produced within a single interaction turn.

Critical Source Files for Multi-Tool Handling

Understanding the implementation requires examining these specific files in the cactus-compute/needle repository:

• needle/agent/tools.py – Houses extract_tool_calls(), the ToolResultRewriter class, and the TOOLS_REGISTRY dictionary.
• needle/agent/fetch.py – Implements run_tool_queue() and manages the async dispatch mechanism.
• needle/cli.py – Contains prompt engineering that instructs the LLM to use the <<tool:NAME>> syntax.
• tests/test_tools.py – Validates correct parsing and multi-tool execution scenarios.

Summary

• Needle parses multiple tool calls from a single LLM response using regex-based extraction of <<tool:NAME>> markers in needle/agent/tools.py.
• Async batch dispatch via run_tool_queue() in needle/agent/fetch.py executes tools concurrently without blocking the main thread.
• Atomic processing ensures individual tool failures convert to error placeholders rather than aborting the entire workflow.
• ToolResultRewriter reintegrates outputs into the original response while adding metadata comments to preserve context.
• Declarative syntax defined in needle/cli.py prompts makes tool invocation deterministic and extensible through the TOOLS_REGISTRY.

Frequently Asked Questions

How does Needle parse multiple tool calls from a single LLM response?

Needle uses the extract_tool_calls() function in needle/agent/tools.py to scan the generated text for blocks wrapped in <<tool:NAME>> and <<end>> markers. A regular-expression matcher collects these blocks in the order they appear, returning a structured list of tool names and arguments for downstream processing.

What happens if one tool fails during a multi-tool execution?

The architecture implements atomic queue processing where each tool runs in isolation. If a tool raises an exception, run_tool_queue() catches the error and inserts an error placeholder into the results list, allowing subsequent tools in the dispatch queue to continue executing normally.

Can developers add custom tools to Needle's registry?

Yes. Developers register new tools by adding a callable to the TOOLS_REGISTRY dictionary in needle/agent/tools.py. The callable must accept the parameters specified in the tool-call syntax, after which it becomes available for single-turn multi-tool workflows immediately.

How does Needle maintain context across multiple tool executions?

The ToolResultRewriter class injects metadata comments (formatted as <!-- tool:NAME result -->) into the response alongside the actual tool output. These comments serve as reference points that the LLM can access in subsequent reasoning steps without requiring additional tool invocations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}