Where Is the Needle 2 Inference Engine Cached and How Is Its Path Resolved?

The Needle 2 inference engine is stored in a versioned cache directory under ~/.cache/cactus-needle/<engine-version>/, with its location determined by a four-step resolution chain defined in needle/__init__.py.

When working with the cactus-compute/needle repository, understanding where the Needle 2 inference engine is cached and how its path is resolved is essential for offline deployments and custom installations. The library uses a deterministic fallback hierarchy to locate the native binary, ensuring reproducible behavior across environments while allowing flexible overrides for advanced use cases.

Cache Location and Resolution Hierarchy

The Four-Step Resolution Order

Inside needle/__init__.py, the private helper _library_path() implements the following lookup sequence:

1. Environment variable override – If NEEDLE_LIB_PATH is set, the function returns that path immediately without further checks.

2. Local package copy – The code checks for a library file matching fetch._lib_name() in the same directory as needle/agent/fetch.py. If present, this local copy takes precedence over the cache.

3. Cache directory lookup – The function constructs the path:

   os.path.join(os.path.expanduser("~"), ".cache", "cactus-needle", fetch.ENGINE_VERSION)

   If the engine binary exists here, that path is returned. As defined in needle/agent/fetch.py, the current ENGINE_VERSION is "2.0.3".

4. On-demand download – If the binary is missing, the code creates the cache directory (if necessary) and invokes fetch.fetch_library() to download the engine from Hugging Face into the cache location.

Key Implementation Files

Path Resolution in needle/init.py

The _library_path() function spans lines 19–31 in needle/__init__.py. This method orchestrates the resolution logic and handles the fallback from environment variables through local copies to the final cache-based storage.

Version Definition in needle/agent/fetch.py

The cache subdirectory name derives from the ENGINE_VERSION constant defined at line 8 of needle/agent/fetch.py:

ENGINE_VERSION = "2.0.3"

This ensures that different engine versions reside in isolated cache directories, preventing binary incompatibility during upgrades.

CLI Integration in needle/cli.py

For air-gapped environments, the needle fetch command (implemented in needle/cli.py, lines 177–245) triggers the same caching logic manually, allowing administrators to pre-download the engine before deployment.

Practical Usage Examples

Default Automatic Caching

When you instantiate the Needle class without configuration, the engine downloads automatically on first use:

from needle import Needle

agent = Needle(tools=my_tools, system="You are a helpful assistant")

Using a Custom Engine Path

For offline devices or custom builds, override the resolution chain by setting the environment variable:

import os
from needle import Needle

os.environ["NEEDLE_LIB_PATH"] = "/opt/needle/needle_engine.so"
agent = Needle(tools=my_tools)

Pre-fetching for Air-Gapped Systems

To populate the cache without instantiating the class, use the CLI:

needle fetch

This executes the download logic directly, placing the binary in ~/.cache/cactus-needle/2.0.3/ for subsequent Python sessions.

Summary

• The Needle 2 inference engine resides in ~/.cache/cactus-needle/<version>/, where version is currently 2.0.3.
• Resolution follows a strict hierarchy: NEEDLE_LIB_PATH → local package copy → existing cache → download via fetch.fetch_library().
• The _library_path() helper in needle/__init__.py (lines 19–31) implements this four-step lookup.
• Version isolation prevents conflicts between different engine releases.
• Use needle fetch to pre-populate the cache on disconnected systems.

Frequently Asked Questions

How do I change where Needle stores the inference engine?

Set the NEEDLE_LIB_PATH environment variable to an absolute path containing the engine binary. When this variable is present, the _library_path() function bypasses all other resolution steps and uses your specified location.

Can Needle run completely offline?

Yes. Pre-populate the cache by running needle fetch while connected, then transfer the ~/.cache/cactus-needle/ directory to your air-gapped environment. Alternatively, set NEEDLE_LIB_PATH to point to a manually copied binary.

What happens when the engine version updates?

The ENGINE_VERSION constant in needle/agent/fetch.py increments, causing _library_path() to look in a new subdirectory (e.g., ~/.cache/cactus-needle/2.0.4/). The library treats each version as independent, leaving existing caches intact while downloading the new binary.

Where does the download originate if the cache is empty?

When resolution reaches step four, fetch.fetch_library() retrieves the engine from Hugging Face, specifically targeting the release assets matching the ENGINE_VERSION string defined in the source.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}