# How to Persist Tool Embeddings Using tool_index_path in Needle

> Persist tool embeddings in Needle using tool_index_path. Save computed vectors to disk and load pre-calculated representations across sessions, avoiding regeneration.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-04

---

**The `tool_index_path` parameter in Needle enables persistent storage of tool embeddings by writing computed vectors to disk, allowing agents to load pre-calculated representations across sessions without regenerating them.**

The open-source Needle framework (cactus-compute/needle) provides a durable indexing mechanism for tool-augmented language models. By configuring the `tool_index_path` parameter, you create a reusable on-disk database that stores vector representations of your tools, eliminating redundant computation during inference and enabling sharing across distributed worker processes.

## How tool_index_path Works

### Lazy Loading and Initialization

When initializing an `Agent`, the framework accepts a `tool_index_path` argument that defaults to `~/.needle/tool_index.db`. The constructor instantiates a `ToolIndex` object that lazily checks for an existing file at this location. If present, the `ToolIndex.load()` method deserializes stored embeddings—using either `numpy.load` or `sqlite3` depending on the backend—and re-instantiates the corresponding `Tool` objects, making them immediately available for model inference.

### Embedding Generation and Persistence

For tools not yet present in the index, Needle computes embeddings by passing the tool's description through the model's tokenizer and encoder (internally referenced via `model/embed`). These vectors are flushed immediately to the specified path, ensuring durability across process restarts. The system uses a write-on-first-use pattern: when a new tool is encountered, its embedding is generated and persisted before the agent executes the tool, guaranteeing that subsequent runs load the vector directly from disk rather than recomputing it.

### Cross-Process Sharing

The persistence layer extends beyond single-process boundaries. As implemented in [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py), the `tool_index_path` propagates to all spawned worker subprocesses, allowing multiple agents to share the same on-disk index. This architecture supports high-throughput deployments and multi-node configurations where the index resides on a shared network volume, ensuring consistent tool representations across your compute cluster.

## Source Code Implementation

The embedding persistence mechanism spans four critical components in the Needle codebase:

**[`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)** defines the `Tool` class and the `ToolIndex` helper that manages read/write operations. This module contains the serialization logic that converts embedding tensors into SQLite or JSON format, handling both the `load()` and save operations.

**[`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)** parses the `--toolindexpath` command-line flag (the CLI equivalent of `tool_index_path`) and forwards the value to the `Agent` constructor, enabling persistent indexes in shell workflows without requiring code changes.

**[`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py)** receives the tool index path during worker initialization and passes it to subprocesses, ensuring that workers reuse the parent's embedding cache rather than maintaining isolated in-memory copies.

**[`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py)** integrates the loaded `ToolIndex` into the inference pipeline, fetching persisted tool embeddings during model execution to inform the model's tool-selection attention mechanism.

## Practical Usage Examples

### Command-Line Configuration

Specify a custom index location when running Needle from the terminal:

```bash
needle run --toolindexpath /data/needle/tool_index.db \
           --model llama-2-7b \
           --prompt "You have access to a calculator tool."

```

### Programmatic Agent Initialization

Create an `Agent` with a persistent index in Python:

```python
from needle.agent import Agent

# Initialize with a custom tool index path

agent = Agent(
    model_name="llama-2-7b",
    toolindexpath="/var/tmp/needle_tool_index.json"
)

@agent.tool
def add(a: int, b: int) -> int:
    """Return the sum of a and b."""
    return a + b

# Embedding is computed and saved on first use

result = agent.run("What is 12 + 34?")

```

### Reloading Existing Embeddings

Subsequent sessions automatically load persisted embeddings:

```python
from needle.agent import Agent

# Reuse the index created in a previous session

agent = Agent(
    model_name="llama-2-7b",
    toolindexpath="/var/tmp/needle_tool_index.json"
)

# The 'add' tool embedding is loaded from disk automatically

print(agent.run("Use the add tool to compute 7+9."))

```

## Summary

- The `tool_index_path` parameter controls the on-disk location for tool embeddings, defaulting to `~/.needle/tool_index.db`.
- The `ToolIndex` class in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) manages lazy loading and immediate persistence of embedding vectors using SQLite or JSON backends.
- Embeddings are generated once per tool and reused across sessions, eliminating redundant computation during inference.
- Worker processes share the same index file via [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py), enabling consistent tool representations in distributed deployments.
- The system writes embeddings immediately after generation when a tool is first used, ensuring the persistent state remains synchronized with tool definitions.

## Frequently Asked Questions

### What file format does tool_index_path use?

Needle supports both SQLite databases (`.db`) and plain-text JSON files for the embedding index. The specific backend format depends on your configuration, but both ensure that tool vectors persist across process restarts and system reboots without manual intervention.

### Can multiple Agent instances share the same tool_index_path?

Yes. Needle propagates the `tool_index_path` to all worker subprocesses through [`needle/_worker.py`](https://github.com/cactus-compute/needle/blob/main/needle/_worker.py), allowing multiple agents to read from and write to the same index file concurrently. For distributed deployments, you can mount the path on a network volume to enable multi-node sharing.

### What is the default location for the tool index?

When `tool_index_path` is not specified, the parameter defaults to `~/.needle/tool_index.db` in the user's home directory. Needle automatically creates this file the first time a tool requires embedding persistence.

### When are new embeddings written to the index?

New embeddings are written immediately after they are computed when a tool is first used. This ensure-on-first-use strategy guarantees that the persistent index contains the most recent embeddings without requiring explicit save calls, while avoiding redundant writes for existing tools.