# How to Set Up Needle for Offline Use on Air‑Gapped Devices

> Set up Needle for offline use on air gapped devices. Download a single 14MB engine library and run inference without network access. Effortless offline deployment.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-26

---

**Yes, Needle fully supports offline setup for air‑gapped devices by fetching a single 14 MB engine library on a connected machine and copying it to the target device—no network access is required for inference.**

Needle from the `cactus-compute/needle` repository is architected for deployment in restricted environments. The engine bundles all model weights into one platform‑specific binary, eliminating the need for ongoing downloads or separate weight files. This guide walks through the exact steps to prepare, transfer, and run Needle without internet connectivity.

## How Needle's Offline Architecture Works

The Needle runtime depends on **exactly one file**: the engine library (`libneedle.so` on Linux, `.dll` on Windows, or `.dylib` on macOS). According to the source code in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) (lines 88–101), this library is fetched from Hugging Face Hub once and cached locally at `~/.cache/cactus-needle/<engine-version>/`. Everything else—tokenization, inference, tool execution—happens inside that library with no further network calls.

This design differs from typical ML deployments that require:
- Separate tokenizer downloads
- Periodic weight updates
- Remote API calls during inference

Needle's **self‑contained engine** removes all of these requirements.

## Step‑by‑Step Offline Setup

### Step 1: Fetch the Engine on a Connected Machine

Use the CLI to download the engine for your target platform and version.

**Current platform:**

```bash
needle fetch

```

The `fetch` command (implemented in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) lines 75–83) prints the cached path and deployment instructions.

**Specific platform for an air‑gapped target:**

```bash
needle fetch --platform-tag manylinux2014_x86_64 --out /tmp/needle_engine

```

This extracts `libneedle.so` to `/tmp/needle_engine/libneedle.so` without modifying the local cache.

### Step 2: Transfer the Engine to the Air‑Gapped Device

Copy the single library file to any of these locations on the target machine:

| Location | Priority | Notes |
|----------|----------|-------|
| `~/.cache/cactus-needle/<version>/libneedle.*` | Default | Mirrors the cache structure from the fetch machine |
| Inside the `needle/` package directory | **Highest** | Takes precedence over cache; useful for immutable deployments |
| Custom path via `NEEDLE_LIB_PATH` | Override | Explicit environment variable pointing to the file |

You can also transfer the entire Python package offline:

```bash

# On connected machine

pip download cactus-needle -d /tmp/needle_pkg

# On air‑gapped device

pip install --no-index --find-links /tmp/needle_pkg cactus-needle

```

### Step 3: Configure and Run Offline

Set two environment variables to ensure strict offline operation:

```bash
export HF_HUB_OFFLINE=1                    # Prevent any Hugging Face Hub network attempts

export NEEDLE_LIB_PATH=/path/to/libneedle.so  # Optional: force specific engine file

python - <<'PY'
import needle

@needle.tool
def echo(msg: str):
    """Return the same message."""
    return {"msg": msg}

agent = needle.Needle(tools=[echo])
print(agent.run("repeat hello world")["results"])
PY

```

The `HF_HUB_OFFLINE=1` setting ensures that a missing engine fails immediately rather than attempting a download, as noted in the README (lines 7–8). The `NEEDLE_LIB_PATH` variable bypasses cache discovery entirely, loading your specified library directly.

## Complete Offline Deployment Example

**Connected preparation:**

```bash

# Fetch Linux x86_64 engine for version 2.0.3

needle fetch --platform-tag manylinux2014_x86_64 --out ./deploy

# Package for transfer

tar czf needle_offline.tar.gz ./deploy/libneedle.so ./my_app.py

```

**Air‑gapped deployment:**

```bash

# Extract and configure

mkdir -p ~/.cache/cactus-needle/2.0.3
cp ./deploy/libneedle.so ~/.cache/cactus-needle/2.0.3/

# Run with guaranteed offline behavior

export HF_HUB_OFFLINE=1
python my_app.py

```

## Key Source Files for Offline Support

The offline workflow is implemented across these locations in `cactus-compute/needle`:

- **[`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)** (lines 75–83): `fetch` command implementation with `--platform-tag` and `--out` options
- **[`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py)** (lines 88–101): Low‑level download and extraction of platform‑specific wheels
- **[`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md)** (lines 56–65): Official "Offline devices" documentation covering fetch, copy, and configure steps
- **[`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md)** (lines 7–8): High‑level confirmation of offline capability with pointer to [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md)

## Summary

- **Single dependency**: Needle requires only one 14 MB engine library file for complete offline operation
- **Three‑step workflow**: Fetch on a connected machine, copy to the air‑gapped device, optionally set `NEEDLE_LIB_PATH` and `HF_HUB_OFFLINE=1`
- **No inference‑time network calls**: All decoding happens locally inside the engine binary
- **Flexible placement**: Engine can reside in user cache, package directory, or custom path
- **Fail‑fast protection**: `HF_HUB_OFFLINE=1` prevents accidental download attempts

## Frequently Asked Questions

### What exactly needs to be transferred to the air‑gapped device?

Only the engine library file (`libneedle.so`, `.dll`, or `.dylib`)—approximately 14 MB. No separate tokenizer files, configuration JSONs, or weight shards are required. You may also transfer the Python wheel for `cactus-needle` itself if not already installed.

### Does Needle verify or re‑download the engine during runtime?

No. The runtime in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) loads the cached library directly. Version checks and downloads only occur during explicit `needle fetch` operations. Setting `NEEDLE_LIB_PATH` skips even the cache lookup.

### Can I use Needle offline with multiple engine versions?

Yes. The cache structure uses version‑specific directories (`~/.cache/cactus-needle/<version>/`). Specify the desired version with `needle fetch --version <x.y.z>` and deploy matching libraries to each target environment.

### What happens if the engine file is missing or corrupted on an offline device?

With `HF_HUB_OFFLINE=1`, the import or initialization raises an immediate error indicating the library cannot be found. Without that setting, the code may attempt to contact Hugging Face Hub and fail with a network timeout. Always set `HF_HUB_OFFLINE=1` for predictable failure modes in air‑gapped deployments.