How Needle's Tool Retrieval Mechanism Works When More Than Five Tools Are Declared

Needle handles more than five tools by sending only the first five tool schemas initially, then dynamically retrieving additional tools on demand when the model requests them.

Needle is an open-source agent framework that enables language models to invoke Python functions via OpenAI-style function calling. When your agent declares more than five tools, Needle implements a sophisticated tool retrieval mechanism that respects the OpenAI API's five-tool-per-request limit while still giving the model access to your entire toolbox. This article explains exactly how this works according to the cactus-compute/needle source code.

The Five-Tool Limit and Why It Matters

OpenAI's function-calling API restricts each request to a maximum of five tool definitions. This isn't arbitrary—large tool schemas consume context window tokens, and too many options can confuse the model's decision-making. Needle's retrieval mechanism turns this constraint into a feature by intelligently batching and fetching tools as needed.

How Tool Registration Works

Every function decorated with @tool becomes available to the language model. The registration process happens in two stages:

Schema Generation

The @tool decorator in needle/agent/tools.py (lines 68–71) automatically builds a JSON schema for each function:

from needle.agent.tools import tool, Field

@tool
def translate(text: str, target_lang: str = Field(enum=["es", "fr", "de"])) -> str:
    """Translate *text* into the target language."""
    return f"Translated: {text}"

The decorator attaches the schema to the function as fn._needle_tool:

# Internal representation created by the decorator

translate._needle_tool == {
    "name": "translate",
    "description": "Translate *text* into the target language.",
    "parameters": {
        "type": "object",
        "properties": {
            "text": {"type": "string"},
            "target_lang": {"enum": ["es", "fr", "de"], "type": "string"}
        },
        "required": ["text"]
    }
}

Global Registry Collection

Decorated functions are automatically collected into a global registry maintained in needle/agent/__init__.py. This registry is a simple list that holds all available tool schemas for the current session.

The Tool Retrieval Workflow

When your agent runs with more than five declared tools, Needle follows a specific batch-and-retrieve pattern:

Step 1: Initial Request with First Five Tools

The agent sends only the first five tool schemas from the registry in the initial request:

from needle.agent import run

# Internally, run() selects tools[0:5] for the first API call

response = run(prompt="Translate 'hello' to Spanish", tools=all_registered_tools)

This respects the OpenAI limit while getting the conversation started.

Step 2: Detecting Missing Tool Requests

If the model's response includes a function_call for a tool not in the current batch, Needle triggers dynamic retrieval. The model essentially "asks" for a tool it knows about but hasn't seen yet.

Step 3: Retrieval and Follow-up Request

Needle performs a tool retrieval step:

• It inspects the registry to locate the missing tool's schema by name
• It sends a follow-up request containing just that additional tool (replaced into a ≤5 tool batch)
• The model now sees the requested tool and can proceed

# Pseudo-code representing the internal logic in needle/agent/__init__.py

if response.function_call.name not in [t["name"] for t in current_batch]:
    extra_tool = registry.lookup(response.function_call.name)
    # Rebuild batch: include requested tool, keep others under limit

    new_batch = build_batch_including(extra_tool, max_size=5)
    response = run(prompt=original_prompt, tools=new_batch, context=response.context)

Step 4: Iterative Resolution

The model may require multiple back-and-forth calls to obtain all needed tools. Each iteration respects the five-tool ceiling while eventually surfacing any tool from your arbitrarily large toolbox.

Key Implementation Files

File	Role in Tool Retrieval
needle/agent/tools.py	Defines Field, build_schema, and the tool decorator that creates JSON schemas for each function
needle/agent/__init__.py	Holds the global tool registry and implements the five-tool limit logic with dynamic retrieval
needle/cli.py	Entry point that invokes the agent and passes appropriate tool lists to the LLM

Practical Example: Six Tools in Action

Here's how Needle handles six declared tools:

from needle.agent.tools import tool
from needle.agent import run

@tool
def search(query: str) -> str: ...

@tool
def calculate(expression: str) -> float: ...

@tool
def translate(text: str, lang: str) -> str: ...

@tool
def summarize(text: str) -> str: ...

@tool
def classify(text: str, labels: list) -> str: ...

@tool  # Sixth tool—exceeds the limit

def format_currency(amount: float, currency: str) -> str: ...

# All six are registered; agent starts with first five

response = run("What's 100 USD in EUR?", tools=all_six_tools)

# Model wants 'calculate' and 'format_currency'

# First request: search, calculate, translate, summarize, classify

# Model calls 'calculate' ✓

# Second request (retrieval): format_currency + 4 others

# Model calls 'format_currency' ✓

Design Benefits of Needle's Tool Retrieval

• Token efficiency: Never sends unused tool schemas
• Scalability: No practical limit on declared tools
• Model clarity: Presents focused options per request
• API compliance: Automatically respects provider limits

Summary

• Needle's @tool decorator in needle/agent/tools.py automatically generates JSON schemas for decorated functions
• All tools are stored in a global registry in needle/agent/__init__.py
• The initial request always includes only the first five tool schemas
• When the model requests a missing tool, Needle performs dynamic retrieval by fetching that specific schema
• Iterative calls continue until the model has access to all needed tools
• This mechanism respects OpenAI's five-tool limit while supporting unlimited declared tools

Frequently Asked Questions

What happens if the model requests two tools that were both excluded from the initial batch?

Needle retrieves both tools in the follow-up request as long as the total remains ≤ five. If more than five tools are needed simultaneously, the model must make multiple retrieval rounds, receiving them in batches.

Is the five-tool limit hardcoded in Needle?

The limit aligns with OpenAI's API constraints as implemented in needle/agent/__init__.py. The retrieval logic assumes this ceiling, though the code structure would allow adjustment if provider limits change.

Can I control which five tools appear in the initial request?

Currently, Needle selects the first five tools by registration order. To prioritize specific tools, register them before others in your code. The source code in needle/agent/__init__.py handles selection automatically without manual batch configuration.

Does tool retrieval add latency to agent responses?

Yes, each retrieval step requires an additional API round-trip. However, in practice, models typically need only 1–2 specific tools per turn, so retrieval overhead is minimal compared to the benefit of maintaining a large, organized toolbox.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}