How Needle Handles Tool Retrieval for Large Tool Catalogues

Needle uses Python entry-points combined with lazy, schema-driven loading to discover and retrieve tools from massive catalogues without importing every implementation at startup.

The cactus-compute/needle repository implements an efficient tool retrieval system designed to handle thousands of tools without draining memory or slowing initialization. By leveraging Python's entry-point system and deferred loading mechanisms, Needle ensures that only requested tools are imported and their schemas computed. This architecture makes it ideal for large-scale agent systems where the tool catalogue might contain hundreds or thousands of functions.

Registering Tools with the @tool Decorator

The foundation of Needle's retrieval system starts with the @tool decorator defined in needle/agent/tools.py. When applied to a function, this decorator introspects the signature and docstring to generate a JSON-Schema representation, storing it in the _needle_tool attribute on the function object itself.

This metadata includes the tool name, description, and parameter specifications, making the function self-describing before it ever enters the global catalogue. The decorator ensures that every exposed function carries its own schema definition, eliminating the need for separate configuration files.

Discovery via importlib.metadata Entry-Points

During runtime initialization in needle/__init__.py, Needle calls importlib.metadata.entry_points(group="needle.tools") to discover all registered tools across installed packages. Because this method returns an iterator rather than a list, the system can enumerate arbitrarily large catalogues without loading them into memory all at once.

Packages declare their tools in pyproject.toml or setup.cfg under the needle.tools group, allowing third-party extensions to integrate seamlessly. The iterator-based approach means that even with thousands of entry-points registered, the initial discovery phase maintains constant memory overhead.

Lazy Loading and Schema Caching

Needle implements lazy loading to defer module imports until a tool is actually requested. When the runtime encounters an entry-point, it checks whether the target object already carries the _needle_tool attribute; if not, it invokes build_schema(entry) to generate the schema on demand.

Once computed, Needle caches the schema directly on the function object, ensuring subsequent lookups retrieve the metadata instantly without repeated introspection overhead. This caching mechanism resides in needle/__init__.py and ensures that expensive reflection operations happen only once per tool, even across multiple agent sessions.

Building the Unified Tool Catalogue

Once discovered, tools are aggregated into a unified catalogue dictionary structured as {fn._needle_tool["name"]: fn._needle_tool for fn in TOOLS}, as seen in environment modules like needle/environments/smart_home.py. This structure allows the agent to query, filter, or paginate available capabilities without importing every underlying implementation.

The separation between schema metadata and executable code means the catalogue remains lightweight even when referencing thousands of heavy tool implementations. The actual execution logic resides in needle/agent/fetch.py, which handles the runtime retrieval and invocation of specific tools from this catalogue.

Practical Implementation Example

The following example demonstrates defining a tool, retrieving its schema, and invoking it through the Needle runtime:

# 1. Define a tool and register it via the @tool decorator

from needle.agent.tools import tool

@tool
def greet(name: str) -> str:
    """Return a friendly greeting."""
    return f"Hello, {name}!"

# 2. Retrieve the tool's schema from the global catalogue

import needle

schema = needle.get_tool_schema("greet")
print(schema["description"])      # → "Return a friendly greeting."

print(schema["parameters"])       # → JSON-Schema for the argument `name`

# 3. Invoke the tool through the Needle agent (runtime will load the function lazily)

result = needle.run_tool("greet", {"name": "Alice"})
print(result)                     # → "Hello, Alice!"

The lazy loader in needle/__init__.py only imports the greet module the first time run_tool is called, demonstrating how Needle maintains low startup costs even with extensive tool catalogues installed.

Summary

• Entry-point discovery allows Needle to find tools across all installed packages without requiring manual registration or configuration files.
• Iterator-based enumeration via importlib.metadata.entry_points keeps memory usage constant regardless of catalogue size.
• Lazy loading defers module imports and schema generation until the tool is actually needed, minimizing startup latency.
• Schema caching stores JSON-Schema metadata on the function object itself, preventing repeated introspection overhead.
• Unified catalogue view aggregates tool metadata separately from implementations, enabling efficient filtering and pagination.

Frequently Asked Questions

How does Needle avoid loading all tools at startup?

Needle uses importlib.metadata.entry_points() which returns an iterator rather than a materialized list, allowing the system to discover thousands of tools without importing their modules. The actual module import and schema generation occur only when run_tool is called for a specific tool, as implemented in needle/__init__.py.

What is the performance impact of the @tool decorator?

The @tool decorator in needle/agent/tools.py performs schema introspection once at decoration time, storing the result in the _needle_tool attribute. This upfront cost is minimal compared to runtime introspection, and the cached schema eliminates repeated reflection during catalogue queries.

Can third-party packages register tools with Needle?

Yes, any Python package can expose tools to Needle by declaring entry-points under the needle.tools group in their pyproject.toml or setup.cfg. The runtime automatically discovers these entries during initialization, making the ecosystem extensible without modifying the core repository.

Where are tool schemas stored before the function is imported?

Before import, schemas exist only as entry-point references in package metadata. Once the module is loaded, the schema is generated via build_schema() and cached in the _needle_tool attribute on the function object. This attribute persists for the lifetime of the process, ensuring fast subsequent access.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}