# How Needle Handles Multiple Tool Selection in a Single Turn: Architecture and Implementation

> Learn how Needle handles multiple tool selection in a single turn. Discover its architecture for efficient async operations and result reintegration.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: architecture
- Published: 2026-08-15

---

**Needle enables single-turn multi-tool execution by parsing declarative tool-call markers, dispatching async operations through a centralized queue, and reintegrating results back into the LLM response.**

Needle, an open-source agent framework from `cactus-compute/needle`, allows language models to invoke several tools within one interaction. This architecture eliminates the need for multiple round-trips between the user and the model when solving complex, multi-step tasks. Understanding how Needle handles multiple tool selection single turn reveals a sophisticated three-stage pipeline designed for concurrency and reliability.

## The Single-Turn Multi-Tool Pipeline Architecture

Needle’s agent architecture processes multiple tool selections through a deterministic pipeline implemented across [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) and [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py). The workflow divides responsibilities into distinct phases: extraction, dispatch, and integration.

### Stage 1: Tool-Call Extraction

After the LLM generates a response, the system scans the text for **tool-call markers** using a regex-based parser. The `extract_tool_calls()` function in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) identifies blocks formatted as `<<tool:NAME>>...<<end>>`, collecting all invocations in the order they appear. This declarative syntax, defined in the prompts within [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), ensures deterministic parsing regardless of tool complexity.

### Stage 2: Batch Dispatch via Async Queue

Extracted calls move to [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), where the `run_tool_queue()` function creates a **dispatch queue**. Each entry contains the tool name, arguments, and a reference to the original LLM output. The system processes these entries sequentially within an `asyncio` event loop, executing each tool in an isolated coroutine. This design maintains low round-trip latency while preventing blocking operations from stalling the main thread.

### Stage 3: Result Integration with ToolResultRewriter

As tools complete, the **`ToolResultRewriter`** class (also in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)) injects outputs back into the original response. The rewriter replaces placeholder blocks with concrete results—whether fetched HTML, generated images, or computed data—and appends metadata comments (`<!-- tool:NAME result -->`) to preserve context for downstream reasoning. The final enriched response returns to the user in a single turn.

## Key Design Patterns for Multi-Tool Reliability

Several architectural decisions ensure robust handling when multiple tools execute simultaneously.

**Atomic Queue Processing**

Each tool call processes atomically; failures convert to error placeholders rather than crashing the pipeline. This isolation guarantees that one failed tool does not abort subsequent operations in the queue.

**Extensible TOOLS_REGISTRY**

Tools register in a global dictionary called `TOOLS_REGISTRY` within [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py). Adding new capabilities requires only defining a callable matching the expected signature and registering it in this dictionary, making the system modular and plugin-friendly.

**Concurrency Safety**

All tool invocations run inside an `asyncio` event loop, allowing the agent to scale to dozens of concurrent calls without blocking the main execution thread.

## Implementing Multiple Tool Selection in a Single Turn

Needle’s single-turn capability shines when handling complex requests like fetching content, processing it, and rendering the output. Consider how the system handles a prompt requesting data retrieval, summarization, and markdown formatting:

```python

# Example prompt structure generated for the LLM

prompt = """
Please fetch the latest blog post from https://example.com/blog, 
summarize its content, and render the summary as a markdown table.

<<tool:fetch>>
url=https://example.com/blog
<<end>>

<<tool:summarize>>
text={{fetch_result}}
<<end>>

<<tool:render>>
format=markdown
content={{summarize_result}}
<<end>>
"""

```

When processing this output, the agent executes the following workflow:

```python
from needle.agent.tools import extract_tool_calls, ToolResultRewriter
from needle.agent.fetch import run_tool_queue

# 1. Parse all tool blocks from the LLM output

calls = extract_tool_calls(prompt)

# 2. Execute tools asynchronously in order

results = await run_tool_queue(calls)

# 3. Stitch results back into the original text

final_output = ToolResultRewriter(prompt, results).rewrite()
print(final_output)

```

The resulting `final_output` contains the fetched HTML, the processed summary, and the rendered markdown table—all produced within a single interaction turn.

## Critical Source Files for Multi-Tool Handling

Understanding the implementation requires examining these specific files in the `cactus-compute/needle` repository:

- **[`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)** – Houses `extract_tool_calls()`, the `ToolResultRewriter` class, and the `TOOLS_REGISTRY` dictionary.
- **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)** – Implements `run_tool_queue()` and manages the async dispatch mechanism.
- **[`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)** – Contains prompt engineering that instructs the LLM to use the `<<tool:NAME>>` syntax.
- **[`tests/test_tools.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_tools.py)** – Validates correct parsing and multi-tool execution scenarios.

## Summary

- **Needle parses multiple tool calls** from a single LLM response using regex-based extraction of `<<tool:NAME>>` markers in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py).
- **Async batch dispatch** via `run_tool_queue()` in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) executes tools concurrently without blocking the main thread.
- **Atomic processing** ensures individual tool failures convert to error placeholders rather than aborting the entire workflow.
- **ToolResultRewriter** reintegrates outputs into the original response while adding metadata comments to preserve context.
- **Declarative syntax** defined in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) prompts makes tool invocation deterministic and extensible through the `TOOLS_REGISTRY`.

## Frequently Asked Questions

### How does Needle parse multiple tool calls from a single LLM response?

Needle uses the `extract_tool_calls()` function in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) to scan the generated text for blocks wrapped in `<<tool:NAME>>` and `<<end>>` markers. A regular-expression matcher collects these blocks in the order they appear, returning a structured list of tool names and arguments for downstream processing.

### What happens if one tool fails during a multi-tool execution?

The architecture implements **atomic queue processing** where each tool runs in isolation. If a tool raises an exception, `run_tool_queue()` catches the error and inserts an error placeholder into the results list, allowing subsequent tools in the dispatch queue to continue executing normally.

### Can developers add custom tools to Needle's registry?

Yes. Developers register new tools by adding a callable to the `TOOLS_REGISTRY` dictionary in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py). The callable must accept the parameters specified in the tool-call syntax, after which it becomes available for single-turn multi-tool workflows immediately.

### How does Needle maintain context across multiple tool executions?

The `ToolResultRewriter` class injects metadata comments (formatted as `<!-- tool:NAME result -->`) into the response alongside the actual tool output. These comments serve as reference points that the LLM can access in subsequent reasoning steps without requiring additional tool invocations.