# How to Persist Tool Embeddings Across Runs Using `tool_index_path` in Needle

> Persist tool embeddings across runs in Needle using tool_index_path. Cache embeddings to disk for instant loading in future sessions, avoiding recomputation. Learn how to optimize your agent.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-31

---

**Set the `tool_index_path` parameter when creating a `Needle` agent to cache tool embeddings to disk, enabling instant loading in subsequent runs without recomputing embeddings.**

The Needle inference engine uses a lightweight contrastive head to embed tool schemas for retrieval-based tool selection. When you have more than five tools, Needle performs a retrieval step to select the most relevant tools for each turn. Recomputing these embeddings on every startup creates significant latency for large tool catalogs. The `tool_index_path` parameter solves this by persisting embeddings to disk with automatic fingerprint-based versioning.

## How `tool_index_path` Works in the Needle Engine

Needle's Python wrapper passes the `tool_index_path` through to the native engine's initialization routine. Understanding this flow helps you use the feature effectively.

### Constructor Storage and Encoding

In [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 56-72), the `Needle` class constructor accepts `tool_index_path` as a string and stores it as a UTF-8-encoded byte sequence:

```python

# needle/__init__.py – constructor handling

self._tool_index_path = tool_index_path.encode("utf-8") if tool_index_path else b""

```

An empty string disables persistence entirely, forcing the engine to use in-memory embeddings only.

### Native Engine Initialization

The wrapper forwards the encoded path to `needle_init` (lines 95-99):

```python

# needle/__init__.py – engine binding

self._engine = needle_init(
    system,
    json.dumps([t._tool_schema for t in tools]),
    self._tool_index_path,  # persisted index location

)

```

The native engine performs three operations based on the path:

1. **Check**: Verify if a file exists at the specified path
2. **Load or Compute**: Load existing embeddings if fingerprint matches; otherwise compute and store new embeddings
3. **Fingerprint Matching**: Index keys combine tool schema hashes with model version for selective updates

## Fingerprint-Based Selective Updates

The index uses a **fingerprint** derived from both the tool schema and the model version. This design enables partial re-embedding when catalogs evolve.

| Scenario | Behavior |
|----------|----------|
| Same schemas, same model | Instant load from disk |
| Modified tool schema | Re-embed only changed tools |
| Model version change | Re-embed all tools with new model |

As documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) (lines 146-150): *"`tool_index_path` persists the embeddings on disk, keyed by a fingerprint … a matching fingerprint loads instantly, a changed schema re‑embeds only what changed."*

## Practical Implementation Example

The following example demonstrates persistence across multiple agent instantiations:

```python
import needle

# Declare a large tool catalog (>5 tools triggers retrieval)

tools = []
for i in range(20):
    @needle.tool
    def make_echo(i=i):
        def echo(msg: str) -> dict:
            """Return the message unchanged."""
            return {"msg": msg}
        return echo
    tools.append(make_echo())

# First run: embeds all tools and writes index to disk

agent = needle.Needle(
    tools=tools,
    tool_index_path="my_tool_index.idx",  # creates on first run

)
response = agent.run("repeat 'hello' using any tool")

# Subsequent runs: loads embeddings instantly from disk

agent2 = needle.Needle(
    tools=tools,
    tool_index_path="my_tool_index.idx",  # reuses existing index

)
response2 = agent2.run("repeat 'world'")

```

### Performance Characteristics

| Metric | Without Persistence | With `tool_index_path` |
|--------|---------------------|------------------------|
| First startup | Embedding latency | Embedding latency + disk write |
| Restarts | Full re-embedding | Near-instant load |
| Cross-process sharing | Not possible | Shared via filesystem |
| Schema updates | N/A (always recomputed) | Incremental update only |

## File Location and Permissions

The `tool_index_path` accepts any writeable filesystem location. Relative paths resolve against the working directory. Consider these conventions:

- **Development**: `./.needle/tool_index.idx` (project-local, gitignored)
- **Production**: `/var/cache/needle/tool_index.idx` or container volume mount
- **Containerized**: Ensure the path resides on a persistent volume

## Key Source Files

Understanding the implementation requires examining these files:

- **[`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)** – Python wrapper storing `_tool_index_path` and binding to native `needle_init`
- **[`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md)** – API documentation covering persistence semantics
- **[`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)** – Tool decorator and schema generation used before fingerprinting
- **[`tests/test_tools.py`](https://github.com/cactus-compute/needle/blob/main/tests/test_tools.py)** – Unit tests validating tool handling behavior

## Summary

- **`tool_index_path`** enables disk-based persistence of expensive tool embeddings
- The index is **fingerprint-keyed** by schema and model version for selective updates
- Set to a **writable file path** to enable persistence; empty string disables it
- **Subsequent runs** load embeddings instantly, dramatically reducing startup time
- **Schema or model changes** trigger partial re-embedding automatically

## Frequently Asked Questions

### What happens if I change a tool's docstring but not its signature?

The fingerprint includes the complete schema, which incorporates the docstring for embedding purposes. Modifying the docstring changes the fingerprint, triggering re-embedding for that specific tool. The engine preserves embeddings for all unchanged tools.

### Can multiple processes share the same `tool_index_path` file simultaneously?

The underlying native engine handles concurrent access through appropriate file locking. Multiple processes can reference the same path safely; however, simultaneous writes (from processes with different schemas) may cause contention. For production deployments with frequent schema changes, use process-specific paths or implement external coordination.

### Does `tool_index_path` work with cloud storage or network filesystems?

The native engine performs standard file I/O operations. Network filesystems (NFS, EFS) and cloud-mounted volumes (FUSE-based GCS/S3 mounts) function correctly provided they support standard POSIX file semantics. Latency-sensitive applications benefit from local SSD storage for index files.

### How do I invalidate or rebuild the index manually?

Delete the file specified by `tool_index_path`. The engine will detect the missing file on next initialization, compute fresh embeddings, and write a new index. There is no built-in API for explicit invalidation—filesystem deletion is the intended mechanism.