# How Needle Selects the Top 5 Tools Per Turn: A Deep Dive into the Tool Selection Pipeline

> Discover how Needle selects the top 5 tools per turn. Learn about its pipeline for parsing LLM probabilities, sorting candidates, and choosing the best S tools for your next step.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-16

---

**Needle selects the top S tools per turn by parsing token-level probability scores from the LLM's tool call suggestions, sorting candidates in descending order, and keeping the first S entries (default 1, configurable via `NEEDLE_TOP_S`).**

Needle is an open-source agent framework that orchestrates LLM-driven tool use. The core question of how Needle selects the top 5 tools per turn—or any number S—involves a precise pipeline that transforms Python functions into OpenAI-compatible schemas, prompts the model, and filters by probability. This article explains each step using the actual source code from the `cactus-compute/needle` repository.

## How the `@tool` Decorator Prepares Functions for Selection

Every function that Needle can invoke must be decorated with `@tool`. This decorator, defined in **[`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)** (lines 62–64), attaches a JSON schema to the function that the LLM can understand.

```python
def tool(fn):
    fn._needle_tool = build_schema(fn)      # ← creates the JSON schema

    return fn

```

The `build_schema(fn)` call introspects the function's signature, type hints, and docstring to generate an OpenAI-compatible tool definition. This schema includes the function name, parameter types, and descriptions.

### Example: Registering a Search Tool

```python
from needle import tool, Field

@tool
def search(query: str, num_results: int = Field(default=5, ge=1, le=10)):
    """Search the web for *query* and return up to *num_results* results."""
    # ... implementation omitted ...

# Verify the schema was attached

assert hasattr(search, "_needle_tool")
print(search._needle_tool["name"])            # → "search"

print(search._needle_tool["parameters"])      # JSON schema sent to LLM

```

Without this decorator, Needle cannot expose a function to the LLM, and it therefore cannot participate in the top S selection process.

## The Turn Execution Flow in [`fetch.py`](https://github.com/cactus-compute/needle/blob/main/fetch.py)

The actual selection logic resides in **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)**. When `run_turn()` is called, the agent executes a three-phase pipeline:

1. **Prompt construction** — aggregates all tool schemas from registered functions
2. **LLM inference** — sends the prompt and receives candidate tool calls with log-probabilities
3. **Scoring and truncation** — sorts by probability and keeps the top S

### Phase 1: Sending Tool Schemas to the LLM

The agent collects every function's `_needle_tool` attribute and includes these schemas in the chat completion request. The LLM generates **a list of possible tool calls**, where each call carries a token-level probability representing the model's confidence.

### Phase 2: Parsing and Scoring Candidate Calls

After the LLM responds, Needle parses the raw output into structured candidates. Each candidate contains:

- The function name to invoke
- The parsed arguments
- A **probability score** extracted from the model's log-probabilities

### Phase 3: Selecting the Top S Tools Per Turn

Here's where the core selection happens. Needle sorts all candidates by their probability scores in **descending order** and slices the list to retain only the first S entries:

```python
S = int(os.getenv("NEEDLE_TOP_S", 1))   # default: 1, override via environment

top_candidates = sorted(candidates, key=lambda c: c.score, reverse=True)[:S]

```

The `NEEDLE_TOP_S` environment variable controls this threshold without code changes. To select the top 5 tools per turn instead of the default 1:

```python
import os
os.environ["NEEDLE_TOP_S"] = "5"

from needle.agent.fetch import run_turn

result = run_turn(user_input="Analyze these three datasets and summarize findings")

# `result` contains up to 5 tool calls, ordered by LLM confidence

```

## Configuration: Adjusting S Without Code Changes

The `NEEDLE_TOP_S` variable provides runtime flexibility. Common configurations include:

| Value | Use Case |
|-------|----------|
| `1` (default) | Single-tool deterministic workflows |
| `3–5` | Exploring multiple promising paths before committing |
| `10+` | Breadth-first search with heavy post-filtering |

Set this before importing Needle components to ensure [`fetch.py`](https://github.com/cactus-compute/needle/blob/main/fetch.py) reads the updated value:

```bash
export NEEDLE_TOP_S=5
python my_agent.py

```

## Why Probability-Based Selection Matters

Needle's approach differs from simple random sampling or hard-coded priority lists. By respecting the **LLM's own confidence estimates**, the framework:

- Surfaces the most semantically relevant tools given the current conversation context
- Allows emergent behavior where the model "votes" for multiple complementary actions
- Provides transparency via inspectable probability scores

The scores are not post-hoc heuristics—they're the model's internal activation probabilities exposed through the API's `logprobs` feature.

## Summary

- The `@tool` decorator in **[`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)** attaches JSON schemas to functions, making them selectable.
- **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)** orchestrates turn execution: prompting, parsing, sorting by probability, and truncating to the top S.
- **S defaults to 1** but is configurable via the `NEEDLE_TOP_S` environment variable.
- Selection relies on **LLM-provided probability scores**, not external ranking heuristics.
- To select the top 5 tools per turn, set `NEEDLE_TOP_S=5` before calling `run_turn()`.

## Frequently Asked Questions

### How does Needle handle ties when multiple tools have the same probability?

Needle uses Python's stable sort, so ties preserve the original order returned by the LLM. In practice, exact probability collisions are rare due to floating-point precision in log-probability calculations. No additional tie-breaking logic is implemented in the current source.

### Can I change the top S limit per turn programmatically instead of via environment variable?

Currently, `NEEDLE_TOP_S` is read once from the environment when [`fetch.py`](https://github.com/cactus-compute/needle/blob/main/fetch.py) loads. For per-call flexibility, you would need to modify `run_turn()` to accept an optional `top_s` parameter or manipulate `os.environ` immediately before invocation. The repository's CLI entry point in **[`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)** does not expose this as a command-line flag.

### What happens if the LLM suggests fewer than S tools?

Needle returns all available candidates without error. The slice operation `[:S]` gracefully handles shorter lists, so if the model proposes 2 tools and `NEEDLE_TOP_S=5`, both are returned. No padding or placeholder logic is applied.

### Does Needle support weighted scoring beyond raw LLM probabilities?

As of the analyzed version, no. The selection pipeline uses the LLM's log-probabilities directly without learned re-ranking or human-configured weights. Custom scoring would require modifying the candidate sorting logic in **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)**.