# How to Deploy Needle Offline and Use Local Weights: Complete Air-Gapped Guide

> Deploy Needle offline and use local weights on air-gapped machines. This guide shows how to run inference without internet access using cached binaries and self-contained weight archives.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-22

---

**Needle supports fully offline deployment by caching the platform-specific engine binary (`libneedle.so`, `libneedle.dylib`, or `libneedle.dll`) alongside a self-contained `.cact` weight archive, allowing you to run inference on air-gapped machines without internet access.**

Needle by Cactus Compute is a lightweight inference engine distributed as a tiny binary library rather than a traditional Python package. Because the engine is completely weight-agnostic and stores all model parameters in a single portable archive, you can easily deploy Needle offline and use local weights on secure or isolated networks after performing a one-time fetch on an internet-connected device.

## Understanding Needle's Offline Architecture

Needle's strict separation between the runtime engine and model weights makes air-gapped deployment straightforward. The system consists of two discrete components that can be transferred independently to target machines.

### The Engine Binary

The core runtime ships as a native shared library—`libneedle.so` on Linux, `libneedle.dylib` on macOS, or `libneedle.dll` on Windows. This binary handles all inference operations and is platform-specific, but contains no model parameters, keeping it compact and reusable across different weight files.

### The .cact Weight Archive

Model parameters are stored in a single **.cact** archive, which contains everything the engine needs to run a specific model or fine-tuned checkpoint. According to the source code comments in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py), this format is self-contained and consumed directly by the engine during the binding phase without external dependencies.

## Fetching Components on a Connected Machine

Before moving to an offline environment, you must acquire both the engine and weights on a machine with internet access.

### Downloading the Engine with `needle fetch`

The `needle fetch` command downloads the pre-compiled engine for your current platform. Internally, this invokes `fetch_library()` in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), which pulls a wheel from the Hugging Face repository, extracts the shared library, and caches it at `~/.cache/cactus-needle/<engine-version>/`.

```bash
needle fetch

```

This stores the platform-appropriate binary (e.g., `libneedle.so`) in the cache directory for later transfer to the offline system.

### Acquiring Model Weights

You can obtain weights using the `needle download` command to fetch a `.cact` file from Hugging Face (e.g., `Cactus-Compute/needle2/model.cact`), or generate your own through fine-tuning followed by `needle build`.

```bash

# Download pre-trained weights

needle download Cactus-Compute/needle2/model.cact --output ./my_model.cact

# Or after fine-tuning locally

needle finetune --data ./training.jsonl
needle build --checkpoint ./checkpoints/final --output ./my_model.cact

```

## Installing Needle on Offline Systems

Once you have the files on a connected machine, transfer them to your air-gapped device using secure media or sneaker-net.

### Deploying the Shared Library

Copy the engine binary from the cache directory to your target machine. You can place it in the standard cache location `~/.cache/cactus-needle/<engine-version>/` or any directory accessible to your application.

```bash

# On the source machine

cp ~/.cache/cactus-needle/2.0.3/libneedle.so /mnt/offline-transfer/

# On the target offline machine

mkdir -p ~/.cache/cactus-needle/2.0.3/
cp /mnt/offline-transfer/libneedle.so ~/.cache/cactus-needle/2.0.3/

```

### Configuring NEEDLE_LIB_PATH

If you place the library in a non-standard location, set the `NEEDLE_LIB_PATH` environment variable. The runtime checks this variable first when resolving the library path, as implemented in `_library_path()` within [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).

```bash
export NEEDLE_LIB_PATH=/opt/cactus/libneedle.so

```

## Loading Local Weights for Inference

With the engine in place, you can load any `.cact` file locally without network calls.

### Python API Usage

When constructing a `Needle` agent, pass the path to your local `.cact` file via the `weights` parameter. The `_bind()` method in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) (lines 70-90) handles loading the weight blob once per process.

```python
import needle

# Create agent with local weights

agent = needle.Needle(weights="/path/to/my_needle.cact")

# Run inference

response = agent.run("What is the weather in Paris?")
print("Answer:", response["answer"])
print("Tool results:", response.get("results", []))

```

Subsequent agents created without a `weights` argument will reuse the already-loaded model in the same process.

### CLI Usage

The command-line interface accepts the `--weights` flag to specify local archives, as defined in the CLI implementation at [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) (lines 81-84).

```bash

# Run a single query

needle run --checkpoint dummy --query "Tell me a joke" --weights ./my_needle.cact

# Start the playground server

needle playground --weights ./my_needle.cact --port 7860

```

## Complete Offline Workflow Example

Follow this pattern to deploy Needle in secure environments:

1. **On the connected machine**: Run `needle fetch` to cache the engine, then `needle download` (or `needle build`) to create a `.cact` file.
2. **Transfer files**: Move `libneedle.so` (or `.dylib`/`.dll`) and the `.cact` archive to the offline machine via secure media.
3. **Configure environment**: Optionally set `NEEDLE_LIB_PATH` if not using the default cache location.
4. **Execute inference**: Use the Python API or CLI commands pointing to your local weight file.

```bash

# Offline machine setup

export NEEDLE_LIB_PATH=/opt/needle/libneedle.so

# Verify offline operation

python -c "import needle; agent=needle.Needle(weights='./model.cact'); print(agent.run('Hello')['answer'])"

```

## Summary

- Needle separates the inference engine from model weights, enabling true offline deployment.
- The engine binary (`libneedle.so`, `libneedle.dylib`, or `libneedle.dll`) is fetched via `fetch_library()` in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) and can be cached for transfer to air-gapped systems.
- Model parameters reside in self-contained **.cact** archives created by `needle build` or downloaded via `needle download`.
- Set `NEEDLE_LIB_PATH` to specify non-standard library locations, handled by `_library_path()` in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).
- Load local weights via the `weights` parameter in Python or `--weights` flag in CLI, with binding logic in `_bind()` at [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py).

## Frequently Asked Questions

### Can I deploy Needle on a completely air-gapped server?

Yes. Because the Needle engine is weight-agnostic and all model parameters are contained in the `.cact` archive, you can transfer both files to an offline machine after fetching them on a connected device. No internet connectivity is required for inference once the files are in place.

### What files do I need to transfer to the offline machine?

You need two files: the platform-specific engine binary (`libneedle.so` for Linux, `libneedle.dylib` for macOS, or `libneedle.dll` for Windows) and the `.cact` weight archive. Optionally, include your application code and any tool definitions required for your use case.

### How does Needle locate the engine library without internet access?

The runtime first checks the `NEEDLE_LIB_PATH` environment variable, then falls back to the platform-specific cache directory (`~/.cache/cactus-needle/<engine-version>/`). This resolution logic resides in `_library_path()` within [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), requiring no network calls to locate the library.

### Can I use multiple different weight files offline?

Yes. Since the engine loads weights dynamically, you can instantiate multiple `Needle` agents with different `weights` parameters, each pointing to a different `.cact` file. The engine only loads the weight blob once per unique path, making it efficient to switch between models on the same offline host.