How to Set Up Needle for Offline Use on Air‑Gapped Devices

Yes, Needle fully supports offline setup for air‑gapped devices by fetching a single 14 MB engine library on a connected machine and copying it to the target device—no network access is required for inference.

Needle from the cactus-compute/needle repository is architected for deployment in restricted environments. The engine bundles all model weights into one platform‑specific binary, eliminating the need for ongoing downloads or separate weight files. This guide walks through the exact steps to prepare, transfer, and run Needle without internet connectivity.

How Needle's Offline Architecture Works

The Needle runtime depends on exactly one file: the engine library (libneedle.so on Linux, .dll on Windows, or .dylib on macOS). According to the source code in needle/agent/fetch.py (lines 88–101), this library is fetched from Hugging Face Hub once and cached locally at ~/.cache/cactus-needle/<engine-version>/. Everything else—tokenization, inference, tool execution—happens inside that library with no further network calls.

This design differs from typical ML deployments that require:

• Separate tokenizer downloads
• Periodic weight updates
• Remote API calls during inference

Needle's self‑contained engine removes all of these requirements.

Step‑by‑Step Offline Setup

Step 1: Fetch the Engine on a Connected Machine

Use the CLI to download the engine for your target platform and version.

Current platform:

needle fetch

The fetch command (implemented in needle/cli.py lines 75–83) prints the cached path and deployment instructions.

Specific platform for an air‑gapped target:

needle fetch --platform-tag manylinux2014_x86_64 --out /tmp/needle_engine

This extracts libneedle.so to /tmp/needle_engine/libneedle.so without modifying the local cache.

Step 2: Transfer the Engine to the Air‑Gapped Device

Copy the single library file to any of these locations on the target machine:

Location	Priority	Notes
~/.cache/cactus-needle/<version>/libneedle.*	Default	Mirrors the cache structure from the fetch machine
Inside the needle/ package directory	Highest	Takes precedence over cache; useful for immutable deployments
Custom path via NEEDLE_LIB_PATH	Override	Explicit environment variable pointing to the file

You can also transfer the entire Python package offline:

# On connected machine

pip download cactus-needle -d /tmp/needle_pkg

# On air‑gapped device

pip install --no-index --find-links /tmp/needle_pkg cactus-needle

Step 3: Configure and Run Offline

Set two environment variables to ensure strict offline operation:

export HF_HUB_OFFLINE=1                    # Prevent any Hugging Face Hub network attempts

export NEEDLE_LIB_PATH=/path/to/libneedle.so  # Optional: force specific engine file

python - <<'PY'
import needle

@needle.tool
def echo(msg: str):
    """Return the same message."""
    return {"msg": msg}

agent = needle.Needle(tools=[echo])
print(agent.run("repeat hello world")["results"])
PY

The HF_HUB_OFFLINE=1 setting ensures that a missing engine fails immediately rather than attempting a download, as noted in the README (lines 7–8). The NEEDLE_LIB_PATH variable bypasses cache discovery entirely, loading your specified library directly.

Complete Offline Deployment Example

Connected preparation:

# Fetch Linux x86_64 engine for version 2.0.3

needle fetch --platform-tag manylinux2014_x86_64 --out ./deploy

# Package for transfer

tar czf needle_offline.tar.gz ./deploy/libneedle.so ./my_app.py

Air‑gapped deployment:

# Extract and configure

mkdir -p ~/.cache/cactus-needle/2.0.3
cp ./deploy/libneedle.so ~/.cache/cactus-needle/2.0.3/

# Run with guaranteed offline behavior

export HF_HUB_OFFLINE=1
python my_app.py

Key Source Files for Offline Support

The offline workflow is implemented across these locations in cactus-compute/needle:

• needle/cli.py (lines 75–83): fetch command implementation with --platform-tag and --out options
• needle/agent/fetch.py (lines 88–101): Low‑level download and extraction of platform‑specific wheels
• doc/apis.md (lines 56–65): Official "Offline devices" documentation covering fetch, copy, and configure steps
• README.md (lines 7–8): High‑level confirmation of offline capability with pointer to doc/apis.md

Summary

• Single dependency: Needle requires only one 14 MB engine library file for complete offline operation
• Three‑step workflow: Fetch on a connected machine, copy to the air‑gapped device, optionally set NEEDLE_LIB_PATH and HF_HUB_OFFLINE=1
• No inference‑time network calls: All decoding happens locally inside the engine binary
• Flexible placement: Engine can reside in user cache, package directory, or custom path
• Fail‑fast protection: HF_HUB_OFFLINE=1 prevents accidental download attempts

Frequently Asked Questions

What exactly needs to be transferred to the air‑gapped device?

Only the engine library file (libneedle.so, .dll, or .dylib)—approximately 14 MB. No separate tokenizer files, configuration JSONs, or weight shards are required. You may also transfer the Python wheel for cactus-needle itself if not already installed.

Does Needle verify or re‑download the engine during runtime?

No. The runtime in needle/agent/fetch.py loads the cached library directly. Version checks and downloads only occur during explicit needle fetch operations. Setting NEEDLE_LIB_PATH skips even the cache lookup.

Can I use Needle offline with multiple engine versions?

Yes. The cache structure uses version‑specific directories (~/.cache/cactus-needle/<version>/). Specify the desired version with needle fetch --version <x.y.z> and deploy matching libraries to each target environment.

What happens if the engine file is missing or corrupted on an offline device?

With HF_HUB_OFFLINE=1, the import or initialization raises an immediate error indicating the library cannot be found. Without that setting, the code may attempt to contact Hugging Face Hub and fail with a network timeout. Always set HF_HUB_OFFLINE=1 for predictable failure modes in air‑gapped deployments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}