How to Set Up Offline/Air-Gapped Deployment with Cached Engines in Needle

Needle's inference engine is a single compiled binary (libneedle.so) that downloads once from Hugging Face, caches locally under ~/.cache/cactus-needle/, and runs entirely offline with no network traffic required once cached.

Deploying large language models in isolated data centers or secure environments requires eliminating external network dependencies. The cactus-compute/needle repository solves this by distributing its core logic as a portable shared library that you can fetch once and replicate anywhere. This guide explains the exact workflow to cache the engine, transfer it to air-gapped machines, and configure Needle to operate completely offline.

Preparing the Engine on a Connected Machine

Needle distributes its inference runtime as a platform-specific binary through Hugging Face Hub. You must download this asset on a machine with internet access before transferring to the isolated environment.

Using the needle fetch Command

The needle fetch command (implemented in needle/cli.py, lines 166-231) handles platform detection and downloads the appropriate libneedle.so binary. By default, it stores the engine in the local cache directory, but you can specify an output directory for easier transfer.

# Download for current platform

needle fetch

# Download for specific target architecture

needle fetch --platform-tag manylinux2014_x86_64 --out /tmp/engine

Understanding the Cache Structure

According to needle/__init__.py (lines 13-28), the _library_path function implements engine discovery logic that looks for binaries under ~/.cache/cactus-needle/<engine-version>/. This directory structure preserves version-specific engines, allowing multiple Needle versions to coexist on the same host.

Transferring Assets to Air-Gapped Environments

Once you have the engine binary, you have three supported methods to make it available on the target machine, as documented in doc/apis.md (lines 55-64) and README.md (lines 23-28).

Method 1: Replicate the Cache Directory

Copy the engine file to the exact same relative path on the air-gapped device. This leverages Needle's default search logic without requiring environment variables.

# Create the cache directory on target

mkdir -p /home/user/.cache/cactus-needle/2.0.0/

# Transfer the binary

scp /tmp/engine/libneedle.so user@airgap:/home/user/.cache/cactus-needle/2.0.0/

Method 2: Direct Drop into Package Directory

Place the binary directly inside the installed needle/ package directory. This location is checked before the user cache and takes precedence automatically, making it ideal for containerized deployments where you control the Python environment.

Method 3: Environment Variable Override

Export NEEDLE_LIB_PATH to force Needle to load a specific file, bypassing all automatic lookup logic. This is the most explicit method for restricted environments.

export NEEDLE_LIB_PATH=/secure/path/libneedle.so

Installing the Python Package Without Network Access

The Python wheel and its dependencies must be transferred separately from the engine binary. On a connected machine, download the artifacts using pip download:

pip download cactus-needle -d /tmp/wheelhouse

Transfer the entire /tmp/wheelhouse directory to the air-gapped host, then perform an offline installation:

pip install --no-index --find-links /tmp/wheelhouse cactus-needle

This --no-index flag guarantees that no network calls are attempted during installation.

Enforcing Strict Offline Mode

To prevent accidental download attempts and ensure deterministic fail-fast behavior in isolated environments, set the Hugging Face Hub offline flag. As documented in doc/apis.md (lines 63-64), HF_HUB_OFFLINE=1 instructs Needle to use only locally cached resources.

export HF_HUB_OFFLINE=1

When this variable is set and the engine is missing from all search paths, Needle raises a clear error immediately rather than attempting to contact Hugging Face.

Complete Deployment Workflow

Combine all steps into a reproducible pipeline for air-gapped deployment:

# 1. On internet-connected machine

pip download cactus-needle -d /tmp/wheelhouse
needle fetch --platform-tag manylinux2014_x86_64 --out /tmp/engine

# 2. Transfer to air-gapped host

scp -r /tmp/wheelhouse user@airgap:/tmp/
scp /tmp/engine/libneedle.so user@airgap:/home/user/.cache/cactus-needle/2.0.0/

# 3. On air-gapped host

export HF_HUB_OFFLINE=1
export NEEDLE_LIB_PATH=/home/user/.cache/cactus-needle/2.0.0/libneedle.so
pip install --no-index --find-links /tmp/wheelhouse cactus-needle
python -c "import needle; print('Engine ready for offline inference')"

Summary

• Single binary architecture: Needle uses one libneedle.so file containing the entire inference engine, downloaded from Hugging Face Hub once and cached locally under ~/.cache/cactus-needle/<version>/.
• Three placement options: Copy to the user cache directory, drop into the needle/ package directory, or specify an exact path via NEEDLE_LIB_PATH.
• Offline package installation: Use pip download on a connected machine and pip install --no-index on the target to eliminate network requirements during setup.
• Safety mechanisms: Set HF_HUB_OFFLINE=1 to guarantee no outbound network calls and ensure clear error messages if the engine is missing from all configured locations.

Frequently Asked Questions

What file exactly do I need to cache for offline Needle deployment?

You need the libneedle.so shared library (or platform equivalent) that corresponds to your target architecture. This single binary, stored in ~/.cache/cactus-needle/<engine-version>/ or referenced via NEEDLE_LIB_PATH, contains the complete inference engine required to run models without internet access.

How does Needle locate the engine binary when running offline?

The _library_path function in needle/__init__.py (lines 13-28) checks locations in strict priority order: first the NEEDLE_LIB_PATH environment variable, then the directory containing the installed needle package, and finally the user cache at ~/.cache/cactus-needle/. If HF_HUB_OFFLINE=1 is set and the file is absent from all locations, it raises a FileNotFoundError instead of attempting a network download.

Can I use needle fetch to download engines for different platforms?

Yes. The needle fetch command accepts a --platform-tag argument (e.g., manylinux2014_x86_64, win_amd64, or macosx_11_0_arm64) allowing you to download binaries for architectures different from your current machine. This is essential when preparing artifacts for air-gapped servers with different CPU or GPU configurations than your workstation.

Will Needle attempt network calls if the cached engine is missing?

Only if HF_HUB_OFFLINE is unset. When HF_HUB_OFFLINE=1 is exported, Needle operates in strict offline mode and will fail immediately with a descriptive error if it cannot locate libneedle.so, guaranteeing that no sensitive air-gapped environment accidentally leaks network traffic to external model repositories.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}