# How Needle Handles Tool Retrieval for Large Tool Catalogues

> Needle efficiently retrieves tools from large catalogues using Python entry points and lazy loading. Discover tools without importing all implementations at startup.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-28

---

**Needle uses Python entry-points combined with lazy, schema-driven loading to discover and retrieve tools from massive catalogues without importing every implementation at startup.**

The `cactus-compute/needle` repository implements an efficient tool retrieval system designed to handle thousands of tools without draining memory or slowing initialization. By leveraging Python's entry-point system and deferred loading mechanisms, Needle ensures that only requested tools are imported and their schemas computed. This architecture makes it ideal for large-scale agent systems where the tool catalogue might contain hundreds or thousands of functions.

## Registering Tools with the @tool Decorator

The foundation of Needle's retrieval system starts with the `@tool` decorator defined in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py). When applied to a function, this decorator introspects the signature and docstring to generate a JSON-Schema representation, storing it in the `_needle_tool` attribute on the function object itself.

This metadata includes the tool name, description, and parameter specifications, making the function self-describing before it ever enters the global catalogue. The decorator ensures that every exposed function carries its own schema definition, eliminating the need for separate configuration files.

## Discovery via importlib.metadata Entry-Points

During runtime initialization in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), Needle calls `importlib.metadata.entry_points(group="needle.tools")` to discover all registered tools across installed packages. Because this method returns an **iterator** rather than a list, the system can enumerate arbitrarily large catalogues without loading them into memory all at once.

Packages declare their tools in [`pyproject.toml`](https://github.com/cactus-compute/needle/blob/main/pyproject.toml) or [`setup.cfg`](https://github.com/cactus-compute/needle/blob/main/setup.cfg) under the `needle.tools` group, allowing third-party extensions to integrate seamlessly. The iterator-based approach means that even with thousands of entry-points registered, the initial discovery phase maintains constant memory overhead.

## Lazy Loading and Schema Caching

Needle implements **lazy loading** to defer module imports until a tool is actually requested. When the runtime encounters an entry-point, it checks whether the target object already carries the `_needle_tool` attribute; if not, it invokes `build_schema(entry)` to generate the schema on demand.

Once computed, Needle caches the schema directly on the function object, ensuring subsequent lookups retrieve the metadata instantly without repeated introspection overhead. This caching mechanism resides in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) and ensures that expensive reflection operations happen only once per tool, even across multiple agent sessions.

## Building the Unified Tool Catalogue

Once discovered, tools are aggregated into a unified catalogue dictionary structured as `{fn._needle_tool["name"]: fn._needle_tool for fn in TOOLS}`, as seen in environment modules like [`needle/environments/smart_home.py`](https://github.com/cactus-compute/needle/blob/main/needle/environments/smart_home.py). This structure allows the agent to query, filter, or paginate available capabilities without importing every underlying implementation.

The separation between schema metadata and executable code means the catalogue remains lightweight even when referencing thousands of heavy tool implementations. The actual execution logic resides in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), which handles the runtime retrieval and invocation of specific tools from this catalogue.

## Practical Implementation Example

The following example demonstrates defining a tool, retrieving its schema, and invoking it through the Needle runtime:

```python

# 1. Define a tool and register it via the @tool decorator

from needle.agent.tools import tool

@tool
def greet(name: str) -> str:
    """Return a friendly greeting."""
    return f"Hello, {name}!"

# 2. Retrieve the tool's schema from the global catalogue

import needle

schema = needle.get_tool_schema("greet")
print(schema["description"])      # → "Return a friendly greeting."

print(schema["parameters"])       # → JSON-Schema for the argument `name`

# 3. Invoke the tool through the Needle agent (runtime will load the function lazily)

result = needle.run_tool("greet", {"name": "Alice"})
print(result)                     # → "Hello, Alice!"

```

The lazy loader in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) only imports the `greet` module the first time `run_tool` is called, demonstrating how Needle maintains low startup costs even with extensive tool catalogues installed.

## Summary

- **Entry-point discovery** allows Needle to find tools across all installed packages without requiring manual registration or configuration files.
- **Iterator-based enumeration** via `importlib.metadata.entry_points` keeps memory usage constant regardless of catalogue size.
- **Lazy loading** defers module imports and schema generation until the tool is actually needed, minimizing startup latency.
- **Schema caching** stores JSON-Schema metadata on the function object itself, preventing repeated introspection overhead.
- **Unified catalogue view** aggregates tool metadata separately from implementations, enabling efficient filtering and pagination.

## Frequently Asked Questions

### How does Needle avoid loading all tools at startup?

Needle uses `importlib.metadata.entry_points()` which returns an iterator rather than a materialized list, allowing the system to discover thousands of tools without importing their modules. The actual module import and schema generation occur only when `run_tool` is called for a specific tool, as implemented in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).

### What is the performance impact of the @tool decorator?

The `@tool` decorator in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) performs schema introspection once at decoration time, storing the result in the `_needle_tool` attribute. This upfront cost is minimal compared to runtime introspection, and the cached schema eliminates repeated reflection during catalogue queries.

### Can third-party packages register tools with Needle?

Yes, any Python package can expose tools to Needle by declaring entry-points under the `needle.tools` group in their [`pyproject.toml`](https://github.com/cactus-compute/needle/blob/main/pyproject.toml) or [`setup.cfg`](https://github.com/cactus-compute/needle/blob/main/setup.cfg). The runtime automatically discovers these entries during initialization, making the ecosystem extensible without modifying the core repository.

### Where are tool schemas stored before the function is imported?

Before import, schemas exist only as entry-point references in package metadata. Once the module is loaded, the schema is generated via `build_schema()` and cached in the `_needle_tool` attribute on the function object. This attribute persists for the lifetime of the process, ensuring fast subsequent access.