# How Needle Handles Tool Retrieval for Large Catalogues Using Contrastive Embeddings

> Needle efficiently retrieves top-5 tools from large catalogues using contrastive embeddings. Discover how Needle indexes and computes similarity for fast, relevant tool selection.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-22

---

**Needle indexes every declared tool schema once at initialization using a native contrastive embedding head, then retrieves only the top-5 most relevant tools per turn by computing similarity between the query embedding and pre-computed tool embeddings.**

The `cactus-compute/needle` repository implements an efficient **tool retrieval** system that enables the framework to scale from a handful of utilities to thousands of tools without overwhelming the LLM's context window. Instead of injecting the entire catalogue into every prompt, Needle employs a contrastive embedding strategy to select relevant tools dynamically. This article examines the technical implementation based on the source code in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) and the API specifications in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md).

## One-Time Indexing of Tool Catalogues

When you instantiate a `Needle` agent with a large tool catalogue, the framework performs an initial indexing phase that embeds each tool's JSON schema exactly once.

### Initialization in needle/__init__.py

The retrieval pipeline begins in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) within the `Needle.__init__` method (lines 54-66). As the constructor processes the tools list, it passes each tool's JSON schema to the native engine through the `needle_init` function. This step transforms the textual schema definitions into dense vector representations that can be compared mathematically.

### The Native needle_init Engine and Contrastive Head

The native engine contains a **built-in contrastive head** that processes tool schemas during initialization. This component embeds every tool schema a single time, creating a persistent vector index of the entire catalogue. According to the API documentation in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), this embedding happens "at init" and leverages contrastive learning techniques to ensure similar tools cluster together in the embedding space.

## Per-Turn Retrieval and the Top-5 Rule

After initialization, Needle employs a **query-driven retrieval** mechanism that runs on every user turn to filter the catalogue down to the most relevant candidates.

### Query Embedding and Similarity Computation

For each user request, Needle first embeds the query text using the same contrastive head. It then computes similarity scores between this query embedding and the pre-computed tool embeddings stored in the index. This similarity calculation identifies which tools in the catalogue are semantically closest to the user's intent.

### Tool Selection and Context Injection

Only the **top-5 scoring tools** are selected and injected into the prompt grammar for that specific turn. As documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), "Five or fewer declared tools render directly. Above that, retrieval engages... only the five highest-scoring tools enter the context." An unselected tool is completely unreachable during that inference step, not merely unlikely to be chosen.

This design represents a deliberate trade-off: five tools fit comfortably inside the model's context window while still providing sufficient capability diversity. Because the retrieval step occurs **outside the language model**, the LLM never processes the full catalogue, keeping token consumption low and latency predictable regardless of catalogue size.

## Persistence and Caching with tool_index_path

Computing embeddings for massive catalogues can be computationally expensive. To optimize repeated runs, Needle provides a `tool_index_path` parameter in `Needle.__init__` that enables disk-based caching of the embedding index.

When you specify a file path for `tool_index_path`, the engine stores the tool embeddings on disk, keyed by a fingerprint that combines the tool schemas and the model version. Subsequent instantiations with the same catalogue and model load the embeddings instantly from this cache. If any tool schema changes, the fingerprint mismatches, triggering a re-embedding of only the modified entries rather than the entire catalogue.

## Code Example: Working with Large Tool Catalogues

The following example demonstrates how to declare a large tool catalogue and leverage the persistent index for optimal performance.

```python
from needle import Needle, tool

# Declare a large catalogue with more than five tools

big_catalog = [
    # Simple functions decorated with @tool

    *[lambda x, i=i: f"Result {i}" for i in range(20)],   # 20 dummy tools

]

# Initialize with persistence to cache embeddings on disk

agent = Needle(
    tools=big_catalog,
    tool_index_path="my_tools.idx",   # ← disk cache for embeddings

)

# Query that only a few tools are relevant to

response = agent.run(
    "Which tool returns the result for i=7?",
    max_steps=1,
)

print(response["function_calls"])

# → Contains only the tool whose embedding best matches the query

```

You can reuse the cached index across separate processes for instant loading:

```python

# New process loads pre-computed embeddings instantly

agent2 = Needle(
    tools=big_catalog,
    tool_index_path="my_tools.idx",   # Loads from cache

)

print(agent2.run("Give me the result for i=13")["function_calls"])

```

## Summary

- **Contrastive embedding**: Needle uses a native contrastive head in `needle_init` to embed tool schemas once during initialization, as implemented in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).
- **Top-5 retrieval**: For each turn, only the five most similar tools to the query embedding enter the context; all others remain unreachable for that step.
- **Persistent indexing**: The `tool_index_path` argument caches embeddings on disk using a fingerprint of schemas and model version, enabling instant reloading and incremental updates.
- **Token efficiency**: By retrieving tools outside the LLM, Needle maintains constant token consumption and latency regardless of catalogue size.

## Frequently Asked Questions

### How does Needle select which tools to include in the context?

Needle computes similarity scores between the embedded user query and pre-computed tool embeddings, then selects exactly the top-5 highest-scoring tools for that turn. This selection happens in the native engine before the LLM processes the request.

### What happens if I declare fewer than five tools?

If your catalogue contains five or fewer tools, Needle disables the retrieval mechanism and renders all tools directly into the context without embedding-based selection, as documented in the "Tool retrieval" section of [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md).

### Can I adjust the number of tools retrieved per turn?

The current implementation in `cactus-compute/needle` hardcodes the limit at five tools. This number represents a fixed design trade-off between context window size and capability diversity, and is not configurable through the public API.

### How does tool_index_path improve startup performance?

The `tool_index_path` parameter stores the embedding index on disk keyed by a fingerprint of your tool schemas and model version. When you instantiate a `Needle` agent with the same catalogue and model, it loads these embeddings instantly rather than recomputing them, significantly reducing initialization time for large catalogues.