How to Set Up Air‑Gapped Offline Deployment for the Needle Engine

Copy the native library to ~/.cache/cactus-needle/<version>/ and set HF_HUB_OFFLINE=1 to run Needle without any network access.

The Needle inference engine is designed as a single, self‑contained native library (~14 MB) that enables complete air‑gapped deployment. Once fetched from Hugging Face, the engine never requires internet connectivity for inference. This guide walks through the exact steps to deploy Needle in restricted environments based on the source code in cactus-compute/needle.

How Needle's Offline Architecture Works

Unlike many ML frameworks that pull models or components at runtime, Needle separates library distribution from inference execution. The Python package cactus-needle contains only bindings and orchestration code. The actual compute kernel—libneedle.so (Linux), libneedle.dylib (macOS), or libneedle.dll (Windows)—is fetched once and cached locally.

According to needle/agent/fetch.py, the engine loader implements a two‑tier lookup:

1. Check the environment variable NEEDLE_LIB_PATH
2. Fall back to ~/.cache/cactus-needle/<ENGINE_VERSION>/

If neither location contains the library, and HF_HUB_OFFLINE is not set, the system attempts a download from the Hugging Face repository Cactus‑Compute/needle2.

Step 1: Download the Engine Library on a Networked Machine

The fetch_library() function in needle/agent/fetch.py handles platform detection, wheel naming, and extraction. Use it to retrieve the correct binary for your target architecture.

from needle.agent.fetch import fetch_library

# ENGINE_VERSION is "2.0.2" as defined in fetch.py

engine_path = fetch_library(
    version="2.0.2",
    dest_dir="/tmp/needle-engine",
    tag=None  # Auto‑detects: manylinux2014_x86_64, macosx_11_0_arm64, etc.

)
print(f"Engine saved to: {engine_path}")

# Output: /tmp/needle-engine/libneedle.so (platform‑specific)

The function constructs the wheel filename using the pattern needle_engine‑{version}‑{py_tag}‑{abi_tag}‑{platform_tag}.whl, then calls hf_hub_download() to fetch and extract only the native library.

Step 2: Transfer the Library to the Air‑Gapped Device

Copy the extracted file to your offline system using any secure transfer method (USB, sneakernet, approved data diode). The destination depends on your deployment preference.

Option A: Standard Cache Location (Recommended)

Place the file in Needle's expected cache directory:

# On the air‑gapped machine

mkdir -p ~/.cache/cactus-needle/2.0.2/
cp libneedle.so ~/.cache/cactus-needle/2.0.2/

The loader checks this path automatically with no additional configuration.

Option B: Bundle with Python Package

Alternatively, place the library inside the installed needle package directory alongside needle/agent/fetch.py. This creates a fully self‑contained installation but requires reinstalling if you upgrade the package.

Step 3: Configure Custom Library Path (Optional)

If your security policy mandates a non‑standard location, set the NEEDLE_LIB_PATH environment variable before importing Needle:

export NEEDLE_LIB_PATH="/opt/secure-libraries/libneedle.so"

import os
os.environ["NEEDLE_LIB_PATH"] = "/opt/secure-libraries/libneedle.so"

import needle  # Loader skips cache, uses explicit path

As implemented in cactus-compute/needle, this bypasses all cache logic and attempts to load the specified file directly.

Step 4: Install Python Dependencies Without Network Access

On your networked machine, collect all required wheels:

pip download cactus-needle --dest ./needle-wheels

Transfer ./needle-wheels/ to the offline host, then install:

pip install --no-index --find-links ./needle-wheels cactus-needle

Enforce Strict Offline Mode

Set HF_HUB_OFFLINE=1 to guarantee that any missing engine triggers an immediate error rather than a failed network attempt. As documented in doc/apis.md, this prevents accidental leakage attempts:

export HF_HUB_OFFLINE=1

import needle

# This raises RuntimeError immediately if libneedle is missing

# instead of hanging on network timeout

agent = needle.Needle()

Step 5: Verify Air‑Gapped Operation

Run a minimal inference test to confirm the engine loads without network activity:

import needle

agent = needle.Needle()
result = agent.run("What is 2 + 2?")
print(result)

Monitor for these success indicators:

• No DNS queries for huggingface.co
• ~28 MB RAM consumption (engine resident set size)
• Sub‑second cold start (no download delays)

If the library is missing, you'll receive a clear error: RuntimeError: Needle engine not found at ....

Complete Deployment Script

#!/usr/bin/env python3
"""End‑to‑end air‑gapped deployment helper."""

import os
import shutil
from pathlib import Path

# Configuration

ENGINE_VERSION = "2.0.2"
SOURCE_ENGINE = "/mnt/transfer/libneedle.so"  # From networked fetch

CACHE_ROOT = Path.home() / ".cache" / "cactus-needle" / ENGINE_VERSION

# Ensure cache directory exists

CACHE_ROOT.mkdir(parents=True, exist_ok=True)

# Install engine library

dest_path = CACHE_ROOT / os.path.basename(SOURCE_ENGINE)
shutil.copy2(SOURCE_ENGINE, dest_path)
print(f"Engine installed: {dest_path}")

# Optional: validate with hash verification

# sha256sum -c needle-engine.sha256

# Verify import works offline

os.environ["HF_HUB_OFFLINE"] = "1"
import needle

agent = needle.Needle()
test_output = agent.run("Verify deployment")
print(f"Offline test passed: {test_output[:50]}...")

Summary

• Fetch once: Use needle.agent.fetch.fetch_library() on a connected machine to download the platform‑ specific libneedle.* from Hugging Face
• Cache locally: Place the library in ~/.cache/cactus-needle/<version>/ or set NEEDLE_LIB_PATH
• Block network: Set HF_HUB_OFFLINE=1 to enforce strict offline operation
• Package offline: Use pip download and pip install --no-index for the Python dependencies
• Verify: Cold‑start inference should complete in under one second with no network traffic

Frequently Asked Questions

What file formats does the Needle engine use per platform?

The engine ships as platform‑specific native libraries: libneedle.so for Linux (glibc‑based), libneedle.dylib for macOS (arm64 and x86_64), and libneedle.dll for Windows. The fetch_library() function automatically selects the correct tag based on platform.machine() and sys.platform as defined in needle/agent/fetch.py.

Can I run Needle offline without ever connecting to Hugging Face?

Yes, if you obtain the engine library through an alternative channel. The NEEDLE_LIB_PATH environment variable accepts any accessible file path, allowing security teams to distribute the binary through approved internal channels rather than hf_hub_download().

How much disk space does an offline deployment require?

The native library occupies approximately 14 MB. The Python package and its dependencies (primarily numpy and typing‑extensions) add roughly 10–20 MB depending on your environment. Total installation footprint is typically under 50 MB.

What happens if the engine version doesn't match the Python package?

Needle validates the engine version at load time. A mismatch between the cached library version and the Python package's expected ENGINE_VERSION constant raises a clear RuntimeError indicating which version was found versus which was required.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}