How to Use Tool Retrieval to Manage a Large Tool Catalogue in Needle

Needle automatically handles large tool catalogues by generating JSON schemas for each tool, deduplicating by name, and letting the inference engine retrieve only the most relevant tools at runtime.

Needle ("cactus-needle") is an open-source framework that treats every external capability as a tool the LLM can invoke. When your system grows to hundreds of possible tools, the library provides built-in mechanisms for schema generation, catalogue management, and dynamic tool selection—eliminating the need for manual filtering or complex retrieval pipelines.

What Is Tool Retrieval in Needle?

Tool retrieval refers to Needle's ability to maintain an exhaustive catalogue of available functions while allowing the inference engine to select and invoke only the tools relevant to a specific user query. This happens dynamically during generation, with the engine emitting function_calls arrays that reference appropriate tools based on context.

The approach differs from static tool selection. Instead of pre-filtering tools before calling the model, Needle passes the complete schema catalogue to the native engine and lets it determine which tools to use at each step of the conversation.

Defining Tools with Automatic Schema Generation

The @tool decorator in needle/agent/tools.py (lines 61-78) attaches a JSON schema to any callable by inspecting its signature, type hints, defaults, and docstring. This schema is stored in the _needle_tool attribute.

from needle import tool, Field

@tool
def send_email(to: str, subject: str, body: str) -> dict:
    """Send an email."""
    return {"status": "sent"}

@tool
def get_weather(location: str) -> dict:
    """Return current weather for a city."""
    return {"location": location, "temp_c": 22}

The build_schema function handles the introspection, converting Python type annotations into valid JSON Schema types and extracting parameter descriptions from the docstring.

Loading and Managing Large Tool Catalogues

Initialising Needle with Hundreds of Tools

The Needle constructor accepts a tools argument that can contain:

• Decorated functions (callables with _needle_tool attribute)
• Pydantic models
• Ready-made schema dictionaries

In needle/__init__.py (lines 88-99), the _resolve method transforms each entry into a standardized schema and stores the callable mapping in self._functions:

import json, pathlib
from needle import Needle

# Load a large catalogue from JSON

catalog_path = pathlib.Path("big_tools.json")
big_catalog = json.loads(catalog_path.read_text())  # list of schema dicts

# Combine file-based and code-based tools

agent = Needle(tools=big_catalog + [send_email, get_weather])

This setup allows you to maintain tool definitions in version-controlled JSON files while keeping frequently-used tools as native Python functions.

Automatic Deduplication by Tool Name

When preparing training data or running inference, Needle ensures that duplicate tool names don't create conflicts. In needle/model/finetune.py (lines 147-155), the _collect_tools function walks through examples and keeps only the first occurrence of each unique name:

# Conceptual flow (from _collect_tools implementation)

seen = set()
deduplicated = []
for tool in all_tools:
    if tool["name"] not in seen:
        seen.add(tool["name"])
        deduplicated.append(tool)

This guarantees that even if your catalogue contains overlapping definitions from multiple sources, the engine receives exactly one schema per tool name.

Runtime Tool Selection and Retrieval

How the Engine Chooses Tools Dynamically

During inference, Needle passes the complete tool catalogue (self._tools_json) to the native engine. The engine then:

1. Analyzes the user query and conversation history
2. Generates a function_calls array containing only the tools needed for the current step
3. Returns control to Needle for execution
4. Re-invokes the model with results, allowing further tool selection

This cycle continues until the engine produces a final response without additional tool calls.

Executing a Multi-Tool Query

# The engine retrieves both tools automatically based on context

response = agent.run("Email the weather forecast to alice@example.com")

print(response["results"])

# → [{'status': 'sent'}]

In this example, the engine identifies that both get_weather and send_email are needed, calls them in sequence, and integrates their outputs.

Organising Tools for Scale

Token Markers for Tool Context

The tokenizer in needle/model/tokenizer.py defines special markers (<tools>, <tool_call>, etc.) that structure how tool information appears in prompts. These markers let the engine distinguish between:

• Available tool definitions (the full catalogue)
• Actual tool invocations (selected at runtime)
• Tool results (returned for context)

Performance Considerations with Large Catalogues

While Needle handles large catalogues gracefully, consider these patterns:

• Split by domain: Maintain separate JSON files for unrelated tool categories, merging only when needed
• Lazy loading: Load tool definitions on first use rather than at import time
• Schema pruning: Remove deprecated parameters from schemas to reduce token usage

Summary

• Schema generation: The @tool decorator in needle/agent/tools.py automatically creates JSON schemas from Python functions using build_schema
• Catalogue resolution: Needle._resolve in needle/__init__.py normalizes mixed tool inputs into consistent schema dictionaries
• Deduplication: _collect_tools in needle/model/finetune.py ensures unique tool names across large catalogues
• Runtime retrieval: The native engine receives complete tool definitions and emits function_calls for only the relevant tools, with Needle.run orchestrating the execution loop

Frequently Asked Questions

How many tools can Needle handle in a single agent?

Needle places no hard limit on tool count. The practical constraint is prompt token budget, as all tool schemas appear in the context window. For catalogues exceeding hundreds of tools, consider domain-specific agents or hierarchical tool grouping to stay within model context limits.

Can I update the tool catalogue after creating a Needle instance?

The Needle class does not expose a public method for adding tools post-initialization. Create a new instance with the updated tools list, or maintain your tools in external storage and reload as needed. The lightweight nature of schema dictionaries makes this operation inexpensive.

Does Needle support semantic search or embedding-based tool retrieval?

The current implementation relies on the native engine's in-context reasoning to select tools from the provided schemas. There is no built-in embedding-based retrieval layer. For very large catalogues where token limits become prohibitive, you would need to implement a pre-filtering step before passing tools to Needle.

What happens if two tools have the same name but different schemas?

The deduplication logic in _collect_tools keeps the first occurrence and silently drops subsequent definitions. To avoid confusion, ensure unique naming across your entire catalogue, or explicitly order your tools list so the preferred definition appears first.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}