Can Needle 2 Run Offline on Air‑Gapped Devices? A Complete Setup Guide

Yes, Needle 2 can run entirely offline on air‑gapped devices once the 14 MB inference engine binary is cached locally.

The cactus-compute/needle repository is engineered for self‑contained deployment, enabling the runtime to execute inference, tool‑calling, and decoding entirely in RAM without external network dependencies. According to the source code in doc/apis.md and the project README, you can deploy Needle 2 on isolated systems by pre‑downloading the engine and configuring environment variables to bypass any network lookups.

How Needle 2 Achieves Offline Operation

Needle 2 operates as a fully standalone inference engine. The system downloads a single 14 MB binary—the engine—to ~/.cache/cactus-needle/v2/ (or v3/ for generation 3) and maps it into RAM for execution.

As implemented in needle/_worker.py, the runtime spawns a separate process that loads the native libneedle.so library from the cache. The needle/model/architecture.py file defines the Simple Attention Network that this binary implements, ensuring all computation happens locally. Once loaded, needle/model/run.py orchestrates tool calling and text generation without initiating HTTP requests, making the system suitable for secure, air‑gapped environments.

Three Methods to Deploy Needle 2 on Air‑Gapped Systems

The documentation outlines three distinct approaches to make the engine available offline.

Method 1: Fetch and Transfer the Engine Binary

On a network‑connected machine, use the CLI to pull the correct platform‑specific binary:

# Download the engine for the current platform

needle fetch --generation 2

# Or specify a target platform for the air‑gapped device

needle fetch --generation 2 --platform-tag manylinux2014_aarch64 --out /tmp/engine

Transfer the cached file to the isolated device via USB or secure file transfer:

scp ~/.cache/cactus-needle/v2/libneedle.so user@offline:/opt/needle/

Method 2: Bundle the Engine with the Package

You can replicate the cached file directly into the installed needle/ package directory on the target device. When libneedle.so resides inside the package folder, it overrides the cache lookup mechanism, allowing the runtime to locate the binary without referencing ~/.cache.

Method 3: Override the Library Path

Set environment variables to point Needle to a custom engine location and force offline mode:

export NEEDLE2_LIB_PATH=/opt/needle/libneedle.so
export HF_HUB_OFFLINE=1  # Prevents any network fallback attempts

For generation 3, use NEEDLE3_LIB_PATH instead. Setting HF_HUB_OFFLINE=1 ensures the loader fails fast with a clear error if the binary is absent, rather than attempting a download.

Installing Needle 2 Without Internet Access

For pure‑Python environments, you can transfer the entire package offline. On a connected machine, download the wheel:

pip download cactus-needle --no-deps -d ./needle-packages

Move the directory to the air‑gapped device and install:

pip install --no-index --find-links ./needle-packages cactus-needle

Verifying Offline Operation

Confirm the setup by running inference with no network interfaces enabled:

import needle

@needle.tool
def echo(msg: str):
    """Return the supplied message."""
    return {"msg": msg}

agent = needle.Needle(tools=[echo])
result = agent.run("echo hello world")
print(result["results"])

When NEEDLE2_LIB_PATH points to a valid libneedle.so and HF_HUB_OFFLINE=1 is set, this script executes completely within the local process space defined in needle/_worker.py, confirming air‑gapped functionality.

Summary

• Needle 2 requires only a single 14 MB download per platform, stored in ~/.cache/cactus-needle/v2/.
• Three deployment methods exist: CLI fetch‑and‑transfer, bundling inside the package directory, or using NEEDLE2_LIB_PATH.
• Environment variables NEEDLE2_LIB_PATH and HF_HUB_OFFLINE=1 eliminate network dependencies.
• Source files needle/_worker.py and needle/model/run.py implement the offline runtime architecture.
• Offline pip installation is supported via pip download and pip install --no-index.

Frequently Asked Questions

Does Needle 2 require an internet connection for inference?

No. Once the engine binary is present on the filesystem, Needle 2 performs all inference, KV cache management, and tool execution locally in RAM. The needle fetch command requires network access, but the runtime itself operates offline according to the implementation in needle/_worker.py.

Where is the Needle 2 engine binary stored?

By default, the system caches the binary in ~/.cache/cactus-needle/v2/libneedle.so for generation 2, or v3/ for generation 3. You can override this location at runtime using the NEEDLE2_LIB_PATH environment variable, or by placing the file inside the installed needle/ package directory.

Can I use Needle 2 on a completely isolated server?

Yes. Air‑gapped deployment is fully supported by pre‑downloading the engine on a connected machine, transferring it to the isolated server, and configuring NEEDLE2_LIB_PATH to point to the binary. Set HF_HUB_OFFLINE=1 to ensure the library loader does not attempt network access.

What happens if the engine binary is missing in offline mode?

If HF_HUB_OFFLINE=1 is set and the binary cannot be found at the configured path or default cache location, the loader raises an immediate error rather than attempting a download. This fail‑fast behavior, documented in doc/apis.md, prevents hangs or timeouts on air‑gapped systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}