How to Configure Needle for Offline or Air-Gapped Deployment

To configure Needle for offline or air-gapped deployment, pre-download the Python package and inference engine on an internet-connected machine, transfer them to the isolated host, and set the NEEDLE_LIB_PATH and HF_HUB_OFFLINE environment variables to force local-only operation.

The cactus-compute/needle repository distributes its inference engine as a downloadable model file that is normally fetched from Hugging Face on first use. For environments that cannot access the internet, you must stage all dependencies and cache files manually before installation.

Pre-Download the Python Package and Dependencies

On a machine with internet access, acquire the Needle wheel and all dependency wheels to avoid runtime network requests. This creates a self-contained installation directory you can transfer to the air-gapped host.

Run the following command to download the package and its dependencies:

pip download cactus-needle -d /tmp/needle_offline

Transfer the resulting directory to your offline host using secure copy or removable media:

scp -r /tmp/needle_offline user@offline-host:/opt/needle_offline

On the air-gapped machine, install Needle using the local wheels without accessing the Python Package Index:

pip install --no-index --find-links /opt/needle_offline cactus-needle

Cache the Inference Engine Ahead of Time

Needle's inference engine must be fetched before entering the offline environment. According to the source code in needle/cli.py, the CLI provides a fetch subcommand that downloads the appropriate engine binary for your platform and prints the cache path.

On the internet-connected machine, run:

needle fetch --out /tmp/needle_cache

This command stores the engine in the specified directory and outputs the cache location (for example, /home/user/.cache/needle/engine-<hash>). Copy this directory to the offline host, preserving the exact directory structure:

scp -r /tmp/needle_cache user@offline-host:/opt/needle_cache

The needle fetch command accepts optional arguments including --platform-tag <tag> to download engines for specific architectures if your online and offline hosts differ.

Configure Environment Variables on the Offline Host

Two environment variables control Needle's offline behavior, as documented in doc/apis.md and llms.txt, and implemented in needle/_worker.py:

• NEEDLE_LIB_PATH: Absolute path to the pre-downloaded engine cache directory
• HF_HUB_OFFLINE: Set to 1 to prevent any network requests to Hugging Face

Configure these variables in your shell profile or application startup script:

export NEEDLE_LIB_PATH=/opt/needle_cache
export HF_HUB_OFFLINE=1

Setting HF_HUB_OFFLINE=1 ensures that if the engine is missing, Needle raises a clear error rather than attempting to download from Hugging Face. The runtime logic in needle/_worker.py checks NEEDLE_LIB_PATH first when loading the engine, bypassing the default cache lookup.

Verify the Offline Configuration

Test the deployment by running an inference command that would normally trigger a download:

needle run --model meta-llama/Meta-Llama-3-8B-Instruct --prompt "Hello, world!"

If the configuration is correct, the command completes without network activity or "engine not found" errors. If Needle reports that the engine is missing, verify that NEEDLE_LIB_PATH points to the exact directory printed by the needle fetch command and that the directory structure was preserved during transfer.

Summary

• Pre-stage dependencies using pip download and install with --no-index on the air-gapped host
• Fetch the engine using needle fetch on an online machine, then copy the cache directory to the offline host
• Set NEEDLE_LIB_PATH to the transferred cache directory so the runtime loads the engine locally
• Set HF_HUB_OFFLINE=1 to disable all network requests and force strict offline mode
• Validate the setup by running a test inference before deploying to production

Frequently Asked Questions

What does the HF_HUB_OFFLINE environment variable do?

Setting HF_HUB_OFFLINE=1 tells the Hugging Face Hub library to operate in offline mode, preventing any network requests to download models or check for updates. When this variable is set and Needle cannot find a required engine locally, it raises an immediate error rather than attempting to fetch from the internet.

Where should I set the NEEDLE_LIB_PATH variable?

Set NEEDLE_LIB_PATH to the absolute path of the directory containing the pre-downloaded engine files transferred from the online machine. This path must match the directory structure created by needle fetch, as the runtime logic in needle/_worker.py searches this specific location before checking default cache directories.

Can I download the Needle engine for a different platform than my online machine?

Yes. The needle fetch command accepts a --platform-tag <tag> argument that allows you to specify the target platform architecture. This is essential when your online staging machine has a different operating system or hardware architecture than your air-gapped production servers.

How do I update Needle in an air-gapped environment?

To update, repeat the entire offline workflow on a machine with internet access: download the new Python wheels with pip download, fetch the latest engine binary with needle fetch, transfer both to the air-gapped network, reinstall the package, and update the NEEDLE_LIB_PATH if the engine directory changed. Always verify the new configuration with a test inference before retiring the previous version.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}