# Where Is the Needle 2 Inference Engine Cached and How Is Its Path Resolved?

> Discover where Needle 2 inference engine is cached at ~/.cache/cactus-needle/<engine-version>/ and understand its path resolution process. Learn how the engine finds its files.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: internals
- Published: 2026-08-30

---

**The Needle 2 inference engine is stored in a versioned cache directory under `~/.cache/cactus-needle/<engine-version>/`, with its location determined by a four-step resolution chain defined in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).**

When working with the cactus-compute/needle repository, understanding where the Needle 2 inference engine is cached and how its path is resolved is essential for offline deployments and custom installations. The library uses a deterministic fallback hierarchy to locate the native binary, ensuring reproducible behavior across environments while allowing flexible overrides for advanced use cases.

## Cache Location and Resolution Hierarchy

### The Four-Step Resolution Order

Inside [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), the private helper `_library_path()` implements the following lookup sequence:

1. **Environment variable override** – If `NEEDLE_LIB_PATH` is set, the function returns that path immediately without further checks.
2. **Local package copy** – The code checks for a library file matching `fetch._lib_name()` in the same directory as [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py). If present, this local copy takes precedence over the cache.
3. **Cache directory lookup** – The function constructs the path:
   ```python
   os.path.join(os.path.expanduser("~"), ".cache", "cactus-needle", fetch.ENGINE_VERSION)
   ```

   If the engine binary exists here, that path is returned. As defined in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), the current `ENGINE_VERSION` is `"2.0.3"`.
4. **On-demand download** – If the binary is missing, the code creates the cache directory (if necessary) and invokes `fetch.fetch_library()` to download the engine from Hugging Face into the cache location.

## Key Implementation Files

### Path Resolution in needle/__init__.py

The `_library_path()` function spans lines 19–31 in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py). This method orchestrates the resolution logic and handles the fallback from environment variables through local copies to the final cache-based storage.

### Version Definition in needle/agent/fetch.py

The cache subdirectory name derives from the `ENGINE_VERSION` constant defined at line 8 of [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py):

```python
ENGINE_VERSION = "2.0.3"

```

This ensures that different engine versions reside in isolated cache directories, preventing binary incompatibility during upgrades.

### CLI Integration in needle/cli.py

For air-gapped environments, the `needle fetch` command (implemented in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), lines 177–245) triggers the same caching logic manually, allowing administrators to pre-download the engine before deployment.

## Practical Usage Examples

### Default Automatic Caching

When you instantiate the `Needle` class without configuration, the engine downloads automatically on first use:

```python
from needle import Needle

agent = Needle(tools=my_tools, system="You are a helpful assistant")

```

### Using a Custom Engine Path

For offline devices or custom builds, override the resolution chain by setting the environment variable:

```python
import os
from needle import Needle

os.environ["NEEDLE_LIB_PATH"] = "/opt/needle/needle_engine.so"
agent = Needle(tools=my_tools)

```

### Pre-fetching for Air-Gapped Systems

To populate the cache without instantiating the class, use the CLI:

```bash
needle fetch

```

This executes the download logic directly, placing the binary in `~/.cache/cactus-needle/2.0.3/` for subsequent Python sessions.

## Summary

- The Needle 2 inference engine resides in `~/.cache/cactus-needle/<version>/`, where version is currently `2.0.3`.
- Resolution follows a strict hierarchy: `NEEDLE_LIB_PATH` → local package copy → existing cache → download via `fetch.fetch_library()`.
- The `_library_path()` helper in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 19–31) implements this four-step lookup.
- Version isolation prevents conflicts between different engine releases.
- Use `needle fetch` to pre-populate the cache on disconnected systems.

## Frequently Asked Questions

### How do I change where Needle stores the inference engine?

Set the `NEEDLE_LIB_PATH` environment variable to an absolute path containing the engine binary. When this variable is present, the `_library_path()` function bypasses all other resolution steps and uses your specified location.

### Can Needle run completely offline?

Yes. Pre-populate the cache by running `needle fetch` while connected, then transfer the `~/.cache/cactus-needle/` directory to your air-gapped environment. Alternatively, set `NEEDLE_LIB_PATH` to point to a manually copied binary.

### What happens when the engine version updates?

The `ENGINE_VERSION` constant in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) increments, causing `_library_path()` to look in a new subdirectory (e.g., `~/.cache/cactus-needle/2.0.4/`). The library treats each version as independent, leaving existing caches intact while downloading the new binary.

### Where does the download originate if the cache is empty?

When resolution reaches step four, `fetch.fetch_library()` retrieves the engine from Hugging Face, specifically targeting the release assets matching the `ENGINE_VERSION` string defined in the source.