Setting Up Needle for Air‑Gapped Offline Environments: Complete Deployment Guide

Needle enables fully offline inference using a single 14 MB engine binary that downloads once and runs forever without network access.

The cactus-compute/needle inference framework is designed specifically for deployment scenarios where internet connectivity is restricted or impossible. Its architecture separates the Python wrapper from the native runtime engine, allowing you to stage the engine binary on a connected machine and transfer only that file to your offline devices. This guide walks through the three‑component workflow—engine fetching, cache management, and offline flags—based on the actual source implementation.

How Needle's Offline Architecture Works

Needle's design centers on a minimal engine fetcher, a cache loader, and environment‑controlled offline flags. Understanding these three pieces clarifies why air‑gapped deployment requires so little effort.

Component	Implementation	Purpose
Engine fetcher	needle/agent/fetch.py (fetch_library function)	Downloads libneedle.so from Hugging Face and stores it in ~/.cache/cactus-needle/<engine-version>/
Cache loader	needle/__init__.py	Resolves engine path on import; falls back to automatic fetch unless HF_HUB_OFFLINE=1 is set
Offline flags	Environment variables (HF_HUB_OFFLINE, NEEDLE_LIB_PATH)	Force network‑free operation or redirect to custom engine locations

The separation is deliberate: after libneedle.so resides on disk, the Python package never initiates HTTP requests. Inference runs entirely in‑process using the loaded shared library.

Step‑by‑Step Air‑Gapped Setup

1. Fetch the Engine on a Connected Machine

Use the CLI sub‑command defined in needle/cli.py (line 163) to retrieve the correct binary for your target platform:

# Default: fetch for current platform

needle fetch

# Cross‑platform: fetch for ARM64 Linux servers

needle fetch --platform-tag manylinux2014_aarch64

The fetch_library function in needle/agent/fetch.py constructs the wheel‑tag URL, downloads from Hugging Face, and prints the cached path:

  fetch    /home/user/.cache/cactus-needle/0.1.0/libneedle.so  downloading from Hugging Face

Key detail: Only this single file—approximately 14 MB—must cross the air gap. Model weights (base or fine‑tuned .cact archives) are handled separately.

2. Package Python Dependencies for Offline Installation

Transfer the Needle package itself without requiring PyPI access during installation:

# On internet‑connected machine

pip download cactus-needle -d ./needle-wheels

# Transfer ./needle-wheels directory to offline target

# Then on air‑gapped device:

pip install --no-index --find-links ./needle-wheels cactus-needle

3. Stage the Engine on the Offline Device

Choose one of three methods supported by the cache loader in needle/__init__.py:

Option A: Standard cache location

mkdir -p ~/.cache/cactus-needle/0.1.0/
cp /path/to/transferred/libneedle.so ~/.cache/cactus-needle/0.1.0/

Option B: Bundled with package installation

# Determine package directory

PKG_DIR=$(python -c "import site, os; print(os.path.join(site.getsitepackages()[0], 'needle'))")
mkdir -p "$PKG_DIR"
cp /path/to/libneedle.so "$PKG_DIR/"

Option C: Custom path via environment variable

export NEEDLE_LIB_PATH=/opt/needle/libneedle.so

The initialization logic in needle/__init__.py checks these locations in order, respecting NEEDLE_LIB_PATH as an override.

4. Execute Inference with Network Disabled

Force guaranteed offline operation using the Hugging Face Hub offline flag:

import os
os.environ["HF_HUB_OFFLINE"] = "1"

import needle

# Load fine-tuned weights if applicable

agent = needle.Needle(weights="/path/to/model.cact", tools=[])
result = agent.run("Summarize this document")
print(result)

With HF_HUB_OFFLINE=1 set, any missing engine triggers an immediate error rather than a network attempt—a safety mechanism for strict air‑gapped compliance.

Environment Variables Reference

Variable	Effect	Typical Use
HF_HUB_OFFLINE=1	Prevents all Hugging Face Hub network calls; errors if engine missing	Mandatory for audited offline environments
NEEDLE_LIB_PATH	Direct path to libneedle.so, bypassing cache search	Custom installation directories, container mounts
HF_HUB_CACHE	Base directory for Hugging Face cache (affects default engine search path)	Shared cache volumes across containers

These variables are documented in the Offline devices section of doc/apis.md in the repository.

Verifying Your Air‑Gapped Installation

Confirm true offline operation by monitoring network activity:

import socket
original_socket = socket.socket

# Block all outbound connections as verification

def guarded_socket(*args, **kwargs):
    raise RuntimeError("Network access attempted")

socket.socket = guarded_socket

import needle
agent = needle.Needle(weights="model.cact")
result = agent.run("Test prompt")  # Succeeds only if fully offline-capable

If inference completes, your deployment is air‑gap verified.

Summary

• Single‑file transfer: Only libneedle.so (~14 MB) must move across the air gap; the Python package and model weights are separate concerns.
• Automatic caching: needle/__init__.py transparently loads cached engines without code changes.
• Fail‑safe offline mode: HF_HUB_OFFLINE=1 guarantees no accidental network leaks.
• Flexible staging: Use standard cache paths, package bundling, or NEEDLE_LIB_PATH for containerized deployments.

The cactus-compute/needle source code implements this workflow through needle/agent/fetch.py for acquisition, needle/__init__.py for resolution, and needle/cli.py for the fetch command—all documented in doc/apis.md.

Frequently Asked Questions

How large is the Needle engine binary that must be transferred to offline devices?

The engine binary (libneedle.so) is approximately 14 MB. This single shared library contains the complete inference runtime. Model weights—whether the base checkpoint or fine‑tuned .cact files—are stored separately and loaded at initialization.

Can I run Needle on a machine that has never connected to the internet?

Yes. Prepare the engine on a connected machine using needle fetch, transfer libneedle.so via approved media, and set NEEDLE_LIB_PATH or place the file in the cache directory. The Python package installs offline using pip install --no-index. No network access is required on the target device.

What happens if the engine is missing and HF_HUB_OFFLINE=1 is set?

The import will raise a FileNotFoundError or similar exception indicating the engine cannot be located. The error originates in needle/__init__.py where cache resolution fails and automatic fetching is disabled by the environment flag. This prevents silent network attempts in audited environments.

Does Needle support cross‑compilation for different architectures?

The fetch system supports cross‑platform retrieval via --platform-tag. Available tags follow Python wheel conventions (e.g., manylinux2014_x86_64, manylinux2014_aarch64). Fetch the appropriate tag on a connected machine, then transfer to matching target hardware. The engine itself is architecture‑specific and cannot be cross‑compiled on‑device.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}