# How to Use Tool Retrieval to Manage a Large Tool Catalogue in Needle

> Learn how to manage a large tool catalogue in Needle. Discover how tool retrieval automatically selects top tools from your repository for efficient management.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-14

---

**Needle automatically handles large tool catalogues by generating JSON schemas for each tool, deduplicating by name, and letting the inference engine retrieve only the most relevant tools at runtime.**

Needle ("cactus-needle") is an open-source framework that treats every external capability as a *tool* the LLM can invoke. When your system grows to hundreds of possible tools, the library provides built-in mechanisms for schema generation, catalogue management, and dynamic tool selection—eliminating the need for manual filtering or complex retrieval pipelines.

## What Is Tool Retrieval in Needle?

Tool retrieval refers to Needle's ability to maintain an exhaustive catalogue of available functions while allowing the inference engine to select and invoke only the tools relevant to a specific user query. This happens dynamically during generation, with the engine emitting `function_calls` arrays that reference appropriate tools based on context.

The approach differs from static tool selection. Instead of pre-filtering tools before calling the model, Needle passes the complete schema catalogue to the native engine and lets it determine which tools to use at each step of the conversation.

## Defining Tools with Automatic Schema Generation

The `@tool` decorator in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) (lines 61-78) attaches a JSON schema to any callable by inspecting its signature, type hints, defaults, and docstring. This schema is stored in the `_needle_tool` attribute.

```python
from needle import tool, Field

@tool
def send_email(to: str, subject: str, body: str) -> dict:
    """Send an email."""
    return {"status": "sent"}

@tool
def get_weather(location: str) -> dict:
    """Return current weather for a city."""
    return {"location": location, "temp_c": 22}

```

The `build_schema` function handles the introspection, converting Python type annotations into valid JSON Schema types and extracting parameter descriptions from the docstring.

## Loading and Managing Large Tool Catalogues

### Initialising Needle with Hundreds of Tools

The `Needle` constructor accepts a `tools` argument that can contain:

- Decorated functions (callables with `_needle_tool` attribute)
- Pydantic models
- Ready-made schema dictionaries

In [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 88-99), the `_resolve` method transforms each entry into a standardized schema and stores the callable mapping in `self._functions`:

```python
import json, pathlib
from needle import Needle

# Load a large catalogue from JSON

catalog_path = pathlib.Path("big_tools.json")
big_catalog = json.loads(catalog_path.read_text())  # list of schema dicts

# Combine file-based and code-based tools

agent = Needle(tools=big_catalog + [send_email, get_weather])

```

This setup allows you to maintain tool definitions in version-controlled JSON files while keeping frequently-used tools as native Python functions.

### Automatic Deduplication by Tool Name

When preparing training data or running inference, Needle ensures that duplicate tool names don't create conflicts. In [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) (lines 147-155), the `_collect_tools` function walks through examples and keeps only the first occurrence of each unique `name`:

```python

# Conceptual flow (from _collect_tools implementation)

seen = set()
deduplicated = []
for tool in all_tools:
    if tool["name"] not in seen:
        seen.add(tool["name"])
        deduplicated.append(tool)

```

This guarantees that even if your catalogue contains overlapping definitions from multiple sources, the engine receives exactly one schema per tool name.

## Runtime Tool Selection and Retrieval

### How the Engine Chooses Tools Dynamically

During inference, Needle passes the complete tool catalogue (`self._tools_json`) to the native engine. The engine then:

1. Analyzes the user query and conversation history
2. Generates a `function_calls` array containing only the tools needed for the current step
3. Returns control to Needle for execution
4. Re-invokes the model with results, allowing further tool selection

This cycle continues until the engine produces a final response without additional tool calls.

### Executing a Multi-Tool Query

```python

# The engine retrieves both tools automatically based on context

response = agent.run("Email the weather forecast to alice@example.com")

print(response["results"])

# → [{'status': 'sent'}]

```

In this example, the engine identifies that both `get_weather` and `send_email` are needed, calls them in sequence, and integrates their outputs.

## Organising Tools for Scale

### Token Markers for Tool Context

The tokenizer in [`needle/model/tokenizer.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/tokenizer.py) defines special markers (`<tools>`, `<tool_call>`, etc.) that structure how tool information appears in prompts. These markers let the engine distinguish between:

- Available tool definitions (the full catalogue)
- Actual tool invocations (selected at runtime)
- Tool results (returned for context)

### Performance Considerations with Large Catalogues

While Needle handles large catalogues gracefully, consider these patterns:

- **Split by domain**: Maintain separate JSON files for unrelated tool categories, merging only when needed
- **Lazy loading**: Load tool definitions on first use rather than at import time
- **Schema pruning**: Remove deprecated parameters from schemas to reduce token usage

## Summary

- **Schema generation**: The `@tool` decorator in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) automatically creates JSON schemas from Python functions using `build_schema`
- **Catalogue resolution**: `Needle._resolve` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) normalizes mixed tool inputs into consistent schema dictionaries
- **Deduplication**: `_collect_tools` in [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) ensures unique tool names across large catalogues
- **Runtime retrieval**: The native engine receives complete tool definitions and emits `function_calls` for only the relevant tools, with `Needle.run` orchestrating the execution loop

## Frequently Asked Questions

### How many tools can Needle handle in a single agent?

Needle places no hard limit on tool count. The practical constraint is prompt token budget, as all tool schemas appear in the context window. For catalogues exceeding hundreds of tools, consider domain-specific agents or hierarchical tool grouping to stay within model context limits.

### Can I update the tool catalogue after creating a Needle instance?

The `Needle` class does not expose a public method for adding tools post-initialization. Create a new instance with the updated `tools` list, or maintain your tools in external storage and reload as needed. The lightweight nature of schema dictionaries makes this operation inexpensive.

### Does Needle support semantic search or embedding-based tool retrieval?

The current implementation relies on the native engine's in-context reasoning to select tools from the provided schemas. There is no built-in embedding-based retrieval layer. For very large catalogues where token limits become prohibitive, you would need to implement a pre-filtering step before passing tools to `Needle`.

### What happens if two tools have the same name but different schemas?

The deduplication logic in `_collect_tools` keeps the first occurrence and silently drops subsequent definitions. To avoid confusion, ensure unique naming across your entire catalogue, or explicitly order your `tools` list so the preferred definition appears first.