# How to Persist Tool Embeddings for Faster Re‑initialization in Needle 2

> Persist tool embeddings in Needle 2 with save and load methods. Store NumPy arrays to disk and reload them to bypass expensive re-creation and speed up initialization.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-02

---

**You can persist tool embeddings in Needle 2 by using the `ToolEmbedding.save()` and `ToolEmbedding.load()` methods, which store NumPy arrays to disk and reload them on subsequent runs to bypass expensive re‑creation.**

Needle 2 generates embeddings for its **tool‑use functions** at startup, and rebuilding these vectors every session adds noticeable latency—especially on slower hardware. The `cactus‑compute/needle` repository includes a built‑in caching mechanism that eliminates this overhead by persisting embeddings to disk. This article walks through the exact implementation paths, environment configuration, and code patterns to enable fast re‑initialization.

## How Tool Embedding Persistence Works in Needle 2

The persistence system operates across five coordinated steps in the source code. Each step maps to specific file paths and method implementations.

### Step 1: Cache Directory Resolution

When Needle is first imported, it resolves where to store embeddings. By default, this is `~/.cache/needle`.

In [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) (lines 67‑71), the library creates this directory if missing:

```python

# Excerpt from needle/agent/fetch.py

CACHE_DIR = Path.home() / ".cache" / "needle"
CACHE_DIR.mkdir(parents=True, exist_ok=True)

```

### Step 2: Embedding Generation

The first time the LLM calls a tool, the **`ToolEmbedding`** class constructs a dense vector from the tool's **name**, **description**, and **JSON schema**.

In [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) (lines 18‑30), this construction parses schemas and tokenizes descriptions—operations that dominate startup time.

### Step 3: Persist Vectors to Disk

After successful generation, `ToolEmbedding.save()` writes the NumPy array to `tool_embeddings.npy` inside the cache directory.

From [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) (lines 140‑152):

```python
def save(self) -> None:
    """Persist embeddings to disk as float32 NumPy array."""
    path = self._cache_path / "tool_embeddings.npy"
    np.save(path, self.vectors.astype(np.float32))

```

### Step 4: Load on Subsequent Startup

On later launches, `ToolEmbedding.load()` checks for the cached file and short‑circuits reconstruction.

In [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) (lines 156‑165):

```python
def load(self) -> bool:
    """Load embeddings from disk if available."""
    path = self._cache_path / "tool_embeddings.npy"
    if path.exists():
        self.vectors = np.load(path)
        return True
    return False

```

### Step 5: Override Cache Location with Environment Variable

Both `save()` and `load()` respect the **`NEEDLE_EMBEDDINGS_DIR`** environment variable for custom paths.

In [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) (lines 23‑30), the resolution logic:

```python
EMBEDDINGS_DIR = os.environ.get(
    "NEEDLE_EMBEDDINGS_DIR",
    str(Path.home() / ".cache" / "needle")
)

```

## Practical Implementation: Enable Embedding Persistence

The [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) entry point automatically triggers `ToolEmbedding.load()` before any LLM interaction. For custom scripts or the Python API, explicitly use the methods below.

### Basic Workflow

```python
from needle.agent.tools import ToolEmbedding

# Automatically loads cached embeddings if available

embedding = ToolEmbedding()
embedding.load()

# If no cache existed, first tool call builds and auto-saves

# Subsequent runs load instantly from disk

```

### Custom Cache Location

```python
import os

os.environ["NEEDLE_EMBEDDINGS_DIR"] = "/my/custom/cache"

from needle.agent.tools import ToolEmbedding

embedding = ToolEmbedding()
found_cache = embedding.load()  # True if loaded from custom path

```

### Force Cache Refresh

After adding new tools or modifying schemas, invalidate the cache:

```python
from needle.agent.tools import ToolEmbedding

embedding = ToolEmbedding()

# Method 1: Use the built-in clear method

embedding.clear_cache()   # Removes tool_embeddings.npy

embedding.load()          # Rebuilds on next tool usage

# Method 2: Manual file deletion

import os
cache_file = os.path.join(os.environ.get("NEEDLE_EMBEDDINGS_DIR", "~/.cache/needle"), "tool_embeddings.npy")
os.remove(os.path.expanduser(cache_file))

```

## Performance Characteristics

| Aspect | Detail |
|--------|--------|
| **Storage format** | NumPy `.npy` file with `float32` arrays |
| **Default location** | `~/.cache/needle/tool_embeddings.npy` |
| **Override mechanism** | `NEEDLE_EMBEDDINGS_DIR` environment variable |
| **Typical speedup** | Eliminates schema parsing and tokenization on every startup |
| **Portability** | `.npy` files are cross‑platform and version‑stable |

The choice of **NumPy's binary format** ensures fast I/O without serialization overhead, and `float32` precision balances embedding quality with storage efficiency.

## Source File Reference

| File | Responsibility |
|------|--------------|
| [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) | Cache path resolution; `NEEDLE_EMBEDDINGS_DIR` handling |
| [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) | `ToolEmbedding` class with `save()`, `load()`, `clear_cache()` |
| [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) | Automatic `ToolEmbedding.load()` invocation at startup |

## Summary

- Needle 2 persists tool embeddings via **`ToolEmbedding.save()`** and **`ToolEmbedding.load()`** in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)
- The default cache directory is `~/.cache/needle`, configurable through **`NEEDLE_EMBEDDINGS_DIR`**
- Embeddings are stored as **NumPy `float32` arrays** in `tool_embeddings.npy` for fast, portable I/O
- Use **`clear_cache()`** or delete the `.npy` file to force regeneration after tool changes
- The CLI entry point auto‑loads cached embeddings, requiring no code changes for standard usage

## Frequently Asked Questions

### What file format does Needle 2 use for persisted embeddings?

Needle 2 uses **NumPy's `.npy` format** with `float32` precision. This provides fast binary I/O without additional dependencies and maintains compatibility across Python versions and operating systems. The implementation in `needle/agent/tools.py:140‑152` calls `np.save()` and `np.load()` directly.

### Can I change where tool embeddings are stored?

Yes. Set the **`NEEDLE_EMBEDDINGS_DIR`** environment variable before importing Needle. The resolution logic in `needle/agent/fetch.py:23‑30` checks this variable first, falling back to `~/.cache/needle` only when unset. Both `save()` and `load()` respect the overridden path.

### Why are my tool embeddings being rebuilt every session?

This occurs when the cache file cannot be found or loaded. Verify that `NEEDLE_EMBEDDINGS_DIR` points to a writable, persistent directory. Ensure the first tool call completes successfully—only then does `ToolEmbedding.save()` trigger. Check `needle/agent/tools.py:156‑165` for the existence check logic.

### How do I force re‑generation of embeddings after adding new tools?

Call **`ToolEmbedding.clear_cache()`** to remove the persisted file, then invoke `ToolEmbedding.load()` to trigger reconstruction during the next tool usage. Alternatively, manually delete `tool_embeddings.npy` from your cache directory.