How the `tool_index_path` Parameter Persists Tool Embeddings in Needle

The tool_index_path parameter specifies the filesystem location where Needle stores serialized NumPy arrays containing tool embeddings, enabling persistent vector indexes that survive across agent restarts without requiring expensive re-computation of semantic representations.

The tool_index_path parameter is central to the embedding persistence strategy in the cactus-compute/needle repository. This parameter instructs the agent where to read and write the vector-based index that represents available tools, ensuring that expensive embedding computations are performed once and reused across subsequent executions.

Core Implementation in needle/agent/tools.py

The primary logic resides in the ToolManager class within needle/agent/tools.py. This class orchestrates the lifecycle of tool embeddings, handling initialization, persistence, and retrieval operations through direct filesystem interaction.

Loading Existing Embeddings at Startup

When initializing the agent, the system checks for an existing file at the path specified by tool_index_path. According to the source code in needle/agent/tools.py (lines 70-85), the implementation attempts to load a previously saved NumPy array using np.load(). If successful, the system rebuilds the in-memory dictionary that maps tool identifiers to their respective embedding vectors, bypassing the need to regenerate embeddings for known tools.

Persisting Updates to Disk

After any modification to the tool registry—whether adding new tools or updating existing metadata—the ToolManager serializes the updated index back to disk. The save logic (lines 120-135 in needle/agent/tools.py) uses numpy.save() to write the embedding array to the tool_index_path location. This operation occurs immediately after index modifications, ensuring that the persistent store remains synchronized with the in-memory state.

Embedding Generation Pipeline

The actual vector generation relies on the shared embedding model defined in needle/model/quantize.py. The embed_tool helper function (lines 150-165) transforms tool descriptions into dense vectors. When tool_index_path points to a non-existent location, the system invokes this helper for each tool, aggregates the results into a NumPy ndarray, and persists this array to the specified path.

CLI Integration in needle/cli.py

The command-line interface exposes this functionality through the --toolindexpath argument. In needle/cli.py (lines 45-55), the parser defines this option and propagates the value to the agent initialization routine.

# Store embeddings in a project-specific cache directory

needle --toolindexpath ./cache/tool_index.npy run my_agent

Practical Usage Patterns

Basic Programmatic Initialization

When instantiating ToolManager directly, pass the persistent path to the constructor:

from needle.agent.tools import ToolManager

# Initialize with a persistent index location

tm = ToolManager(tool_index_path="~/needle_cache/tool_index.npy")

# Register tools - embeddings are computed and cached automatically

tm.register_tool(name="search", description="Google search operation")

Cross-Session Persistence

The persistence mechanism ensures that subsequent agent runs benefit from previous computations:

import os
from needle.agent.tools import ToolManager

path = os.path.expanduser("~/my_project/tool_embeddings.npy")
tool_mgr = ToolManager(tool_index_path=path)

# Load existing embeddings from previous sessions

embedding = tool_mgr.get_embedding("search")

Index Recovery and Reloading

To force a refresh of the index from disk after external modifications:

# Force reload from tool_index_path

tool_mgr.reload_index()

File Format and Serialization

The index file stored at tool_index_path utilizes NumPy's binary format (.npy) for efficient storage and rapid loading. The needle/model/export.py module (lines 240-260) demonstrates similar serialization patterns for embedding tables, confirming the use of standard NumPy serialization primitives rather than custom binary formats. This format choice enables the system to store high-dimensional float32 vectors compactly while maintaining sub-second load times for indexes containing thousands of tools.

Summary

• The tool_index_path parameter specifies where Needle stores serialized tool embeddings as NumPy arrays.
• The ToolManager class in needle/agent/tools.py handles automatic loading (lines 70-85) and saving (lines 120-135) of the index.
• If no file exists at the specified path, the system generates embeddings using the model in needle/model/quantize.py (lines 150-165) and creates a new persistent index.
• The CLI accepts --toolindexpath as defined in needle/cli.py (lines 45-55) for command-line workflows.
• This persistence mechanism avoids redundant API calls to embedding services across agent restarts.

Frequently Asked Questions

What file format does tool_index_path use?

The parameter points to a NumPy binary file (.npy) created via numpy.save(). The serialized data structure contains the vector embeddings that map tool descriptions to their semantic representations. While the system primarily uses NumPy format, the underlying serialization logic in needle/model/export.py suggests JSON equivalents may be supported for specific export scenarios.

When should I specify a custom tool_index_path?

Specify a custom path when running multiple agent instances that should share tool embeddings, or when working across different projects that require distinct tool sets. Pointing to a stable location—such as a project-wide cache directory—ensures that embeddings survive between container restarts or CI/CD pipeline executions, dramatically reducing startup latency by avoiding recomputation.

How does Needle handle corrupted or incompatible index files?

If numpy.load() fails to read the file at tool_index_path—whether due to corruption, format incompatibility, or version mismatch—the ToolManager treats the scenario as a missing index. The system builds a fresh embedding index from scratch using the current tool definitions and overwrites the problematic file with the newly generated data, ensuring the agent remains functional even with damaged persistence files.

Can multiple agents share the same tool_index_path concurrently?

While the index file can be shared across sequential runs, concurrent write operations from multiple agents to the same tool_index_path may result in race conditions or corrupted files. The current implementation in needle/agent/tools.py does not implement file locking mechanisms; therefore, assign unique paths to simultaneously running agents to prevent data corruption.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}