# How to Run Needle 2 Model Offline on Embedded Devices: Complete Deployment Guide

> Learn to run Needle 2 model offline on embedded devices. Deploy libneedle and model weights locally, setting environment variables for seamless network-free inference. Get the complete guide.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-29

---

**Needle 2 runs offline on embedded devices by fetching the native engine library (`libneedle.so/.dylib/.dll`) and model weights (`.cact` checkpoint) in advance, then executing inference with environment variables like `NEEDLE_LIB_PATH` and `HF_HUB_OFFLINE=1` to eliminate network dependencies.**

Needle 2 provides a lightweight inference engine specifically designed for air-gapped and resource-constrained environments. According to the cactus-compute/needle source code, the framework ships as a single binary engine alongside optional checkpoint files, enabling you to run Needle 2 model offline on embedded devices without any runtime network connectivity.

## Prerequisites for Offline Deployment

Before deploying to embedded hardware, you need both the native library and the model weights. The system consists of two primary components: the **engine binary** (`libneedle.so`, `libneedle.dylib`, or `libneedle.dll`) and the **checkpoint file** (`.cact` extension containing model weights). According to [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), these files must be pre-positioned on the target filesystem before disconnecting from the network.

## Three-Stage Offline Deployment Workflow

### Stage 1: Fetch the Native Engine Library

The first step involves pulling the pre-compiled native library for your specific platform. In [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), the `_platform_tag()` function determines the correct wheel tag for your architecture, while `_lib_name_for()` selects the appropriate library name.

Use the CLI command `needle fetch` to download the engine from the Hugging Face repository *Cactus-Compute/needle2*:

```bash
needle fetch

```

This places the library in the local cache at `~/.cache/cactus-needle/<engine-version>/`. For cross-compilation scenarios, specify the platform tag explicitly:

```bash
needle fetch --platform-tag manylinux2014_aarch64

```

### Stage 2: Download the Model Checkpoint

Next, retrieve the model weights using the `download` command implemented in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py). This command calls either `fetch.download_platform` for standard engines or `hf_hub_download` for custom weights.

Download the base checkpoint for your platform:

```bash
needle download linux-x86_64

```

Or fetch a custom tuned file:

```bash
needle download <org>/<repo>/<file>.cact

```

The resulting `.cact` file can reside in the cache path or alongside your application binary.

### Stage 3: Run Inference Without Network Activity

Once the engine and weights are local, execute inference using either the CLI or Python API. Set `NEEDLE_LIB_PATH` to point to the engine binary and optionally set `HF_HUB_OFFLINE=1` to prevent any network attempts.

CLI approach:

```bash
needle run \
    --checkpoint path/to/needle2.cact \
    --query "What is the weather in Paris?" \
    --tools tools.json

```

Python approach:

```python
import os
import needle

# Point to the pre-copied library binary

os.environ["NEEDLE_LIB_PATH"] = "/home/user/.cache/cactus-needle/2.0.3/libneedle.so"

# Optional: ensure complete offline mode

os.environ["HF_HUB_OFFLINE"] = "1"
os.environ["NEEDLE_TELEMETRY"] = "0"

# Load checkpoint - no network requests performed

agent = needle.Needle(weights="my_needle.cact", tools=[...])

# Execute inference

response = agent.run("Dim the living room lights to 30%")
print(response["results"])

```

## Embedded Device Configuration Guide

When deploying to air-gapped embedded systems, copy the cached `libneedle.*` to a persistent location such as `/usr/lib/needle/` or `/opt/needle/`. The engine consumes approximately **28 MiB of RAM**, making it suitable for devices with as little as 64 MiB total memory.

Key environment variables for embedded deployment:

- **`NEEDLE_LIB_PATH`**: Absolute path to the native engine binary (e.g., `/opt/needle/libneedle.so`)
- **`HF_HUB_OFFLINE=1`**: Prevents the Hugging Face hub library from attempting network downloads
- **`NEEDLE_TELEMETRY=0`**: Disables telemetry reporting for completely silent operation

## Platform-Specific Implementation Examples

### Linux ARM64 Embedded Board

For ARM64 microcontrollers running Linux, fetch the specific platform build and execute:

```bash

# Pull engine binary for linux-arm64

needle fetch --platform-tag manylinux2014_aarch64

# Download base checkpoint

needle download linux-arm64

# Run with no external connectivity

needle run --checkpoint ~/.cache/cactus-needle/2.0.3/needle2.cact \
           --query "Tell me a joke"

```

### Python Integration on Resource-Constrained Devices

For direct Python integration on embedded Linux systems:

```python
import os
import needle

# Configure library path for embedded filesystem

os.environ["NEEDLE_LIB_PATH"] = "/opt/needle/libneedle.so"
os.environ["HF_HUB_OFFLINE"] = "1"

# Initialize with pre-downloaded weights

agent = needle.Needle(weights="/opt/needle/needle2.cact", tools=[])

# Single-step inference

print(agent.run("What is the temperature?"))

```

## Summary

- Needle 2 operates as a single binary engine (`libneedle.so/.dylib/.dll`) paired with `.cact` checkpoint files, requiring no network access during inference.
- Use `needle fetch` to download platform-specific engines from [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), specifying tags like `manylinux2014_aarch64` for embedded targets.
- Download weights via `needle download` from [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) before disconnecting from the network.
- Set `NEEDLE_LIB_PATH` to the engine location and `HF_HUB_OFFLINE=1` to guarantee offline operation on air-gapped devices.
- The engine footprint stays under 28 MiB RAM, suitable for embedded devices with >64 MiB memory.

## Frequently Asked Questions

### What files do I need to transfer to an air-gapped device to run Needle 2?

You need two components: the native engine library (`libneedle.so`, `libneedle.dylib`, or `libneedle.dll`) and the model checkpoint file (`.cact` extension). According to the source code in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py), the engine handles platform-specific tagging automatically when you use `needle fetch` on a connected system first.

### How do I prevent Needle 2 from attempting network connections at runtime?

Set the environment variable **`HF_HUB_OFFLINE=1`** before importing the needle module. Additionally, set **`NEEDLE_TELEMETRY=0`** to disable telemetry. As implemented in the source codebase, these flags force the library to operate exclusively with local files and cache directories.

### What is the minimum RAM requirement for running Needle 2 on embedded hardware?

The engine requires approximately **28 MiB** of RAM during inference. The documentation in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) recommends devices with at least 64 MiB total memory to accommodate the engine, weights, and operating system overhead.

### Can I use custom-tuned model weights instead of the base checkpoint?

Yes. Use the command `needle download <org>/<repo>/<file>.cact` to fetch custom weights from Hugging Face Hub. The download logic in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) supports arbitrary repository paths, allowing you to deploy fine-tuned models to offline embedded devices using the same deployment workflow as base models.