# How to Set Up Offline/Air-Gapped Deployment with Cached Engines in Needle > Learn to set up offline/air-gapped deployment with cached engines in Needle. Run Needle inference binaries entirely offline after initial cache download for secure, isolated environments. - Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle) - Tags: how-to-guide - Published: 2026-08-18 --- **Needle's inference engine is a single compiled binary (`libneedle.so`) that downloads once from Hugging Face, caches locally under `~/.cache/cactus-needle/`, and runs entirely offline with no network traffic required once cached.** Deploying large language models in isolated data centers or secure environments requires eliminating external network dependencies. The `cactus-compute/needle` repository solves this by distributing its core logic as a portable shared library that you can fetch once and replicate anywhere. This guide explains the exact workflow to cache the engine, transfer it to air-gapped machines, and configure Needle to operate completely offline. ## Preparing the Engine on a Connected Machine Needle distributes its inference runtime as a platform-specific binary through Hugging Face Hub. You must download this asset on a machine with internet access before transferring to the isolated environment. ### Using the `needle fetch` Command The `needle fetch` command (implemented in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), lines 166-231) handles platform detection and downloads the appropriate `libneedle.so` binary. By default, it stores the engine in the local cache directory, but you can specify an output directory for easier transfer. ```bash # Download for current platform needle fetch # Download for specific target architecture needle fetch --platform-tag manylinux2014_x86_64 --out /tmp/engine ``` ### Understanding the Cache Structure According to [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 13-28), the `_library_path` function implements engine discovery logic that looks for binaries under `~/.cache/cactus-needle//`. This directory structure preserves version-specific engines, allowing multiple Needle versions to coexist on the same host. ## Transferring Assets to Air-Gapped Environments Once you have the engine binary, you have three supported methods to make it available on the target machine, as documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) (lines 55-64) and [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) (lines 23-28). ### Method 1: Replicate the Cache Directory Copy the engine file to the exact same relative path on the air-gapped device. This leverages Needle's default search logic without requiring environment variables. ```bash # Create the cache directory on target mkdir -p /home/user/.cache/cactus-needle/2.0.0/ # Transfer the binary scp /tmp/engine/libneedle.so user@airgap:/home/user/.cache/cactus-needle/2.0.0/ ``` ### Method 2: Direct Drop into Package Directory Place the binary directly inside the installed `needle/` package directory. This location is checked before the user cache and takes precedence automatically, making it ideal for containerized deployments where you control the Python environment. ### Method 3: Environment Variable Override Export `NEEDLE_LIB_PATH` to force Needle to load a specific file, bypassing all automatic lookup logic. This is the most explicit method for restricted environments. ```bash export NEEDLE_LIB_PATH=/secure/path/libneedle.so ``` ## Installing the Python Package Without Network Access The Python wheel and its dependencies must be transferred separately from the engine binary. On a connected machine, download the artifacts using `pip download`: ```bash pip download cactus-needle -d /tmp/wheelhouse ``` Transfer the entire `/tmp/wheelhouse` directory to the air-gapped host, then perform an offline installation: ```bash pip install --no-index --find-links /tmp/wheelhouse cactus-needle ``` This `--no-index` flag guarantees that no network calls are attempted during installation. ## Enforcing Strict Offline Mode To prevent accidental download attempts and ensure deterministic fail-fast behavior in isolated environments, set the Hugging Face Hub offline flag. As documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) (lines 63-64), `HF_HUB_OFFLINE=1` instructs Needle to use only locally cached resources. ```bash export HF_HUB_OFFLINE=1 ``` When this variable is set and the engine is missing from all search paths, Needle raises a clear error immediately rather than attempting to contact Hugging Face. ## Complete Deployment Workflow Combine all steps into a reproducible pipeline for air-gapped deployment: ```bash # 1. On internet-connected machine pip download cactus-needle -d /tmp/wheelhouse needle fetch --platform-tag manylinux2014_x86_64 --out /tmp/engine # 2. Transfer to air-gapped host scp -r /tmp/wheelhouse user@airgap:/tmp/ scp /tmp/engine/libneedle.so user@airgap:/home/user/.cache/cactus-needle/2.0.0/ # 3. On air-gapped host export HF_HUB_OFFLINE=1 export NEEDLE_LIB_PATH=/home/user/.cache/cactus-needle/2.0.0/libneedle.so pip install --no-index --find-links /tmp/wheelhouse cactus-needle python -c "import needle; print('Engine ready for offline inference')" ``` ## Summary - **Single binary architecture**: Needle uses one `libneedle.so` file containing the entire inference engine, downloaded from Hugging Face Hub once and cached locally under `~/.cache/cactus-needle//`. - **Three placement options**: Copy to the user cache directory, drop into the `needle/` package directory, or specify an exact path via `NEEDLE_LIB_PATH`. - **Offline package installation**: Use `pip download` on a connected machine and `pip install --no-index` on the target to eliminate network requirements during setup. - **Safety mechanisms**: Set `HF_HUB_OFFLINE=1` to guarantee no outbound network calls and ensure clear error messages if the engine is missing from all configured locations. ## Frequently Asked Questions ### What file exactly do I need to cache for offline Needle deployment? You need the `libneedle.so` shared library (or platform equivalent) that corresponds to your target architecture. This single binary, stored in `~/.cache/cactus-needle//` or referenced via `NEEDLE_LIB_PATH`, contains the complete inference engine required to run models without internet access. ### How does Needle locate the engine binary when running offline? The `_library_path` function in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 13-28) checks locations in strict priority order: first the `NEEDLE_LIB_PATH` environment variable, then the directory containing the installed `needle` package, and finally the user cache at `~/.cache/cactus-needle/`. If `HF_HUB_OFFLINE=1` is set and the file is absent from all locations, it raises a `FileNotFoundError` instead of attempting a network download. ### Can I use `needle fetch` to download engines for different platforms? Yes. The `needle fetch` command accepts a `--platform-tag` argument (e.g., `manylinux2014_x86_64`, `win_amd64`, or `macosx_11_0_arm64`) allowing you to download binaries for architectures different from your current machine. This is essential when preparing artifacts for air-gapped servers with different CPU or GPU configurations than your workstation. ### Will Needle attempt network calls if the cached engine is missing? Only if `HF_HUB_OFFLINE` is unset. When `HF_HUB_OFFLINE=1` is exported, Needle operates in strict offline mode and will fail immediately with a descriptive error if it cannot locate `libneedle.so`, guaranteeing that no sensitive air-gapped environment accidentally leaks network traffic to external model repositories.