# How to Deploy Needle 2 on Embedded Systems: Complete Guide for Air-Gapped Devices

> Deploy Needle 2 on air-gapped embedded systems with our three-step offline guide. Run inference locally on ~28MB RAM with zero network needs. Get started now.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-09-05

---

**Needle 2 deploys to embedded systems via a three-step offline workflow: cache the ~14 MiB native inference engine locally, transfer a self-contained `.cact` model archive to the target device, and execute inference entirely in-process using either the Python API or the compiled native runner, requiring only 28 MiB of RAM with zero ongoing network connectivity.**

Deploying large language models to resource-constrained hardware traditionally requires complex orchestration and persistent cloud connectivity. The `cactus-compute/needle` repository eliminates these constraints by packaging the entire inference runtime into a single binary that operates in ultra-low-memory environments. This guide explains how to deploy Needle 2 on embedded systems using the official offline deployment strategy documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md).

## System Requirements and Resource Constraints

### Memory and Storage Specifications

**Needle 2** is optimized for microcontrollers and single-board computers with severe resource limits. The native engine binary occupies approximately **14 MiB** of storage, while a full inference session executes within roughly **28 MiB** of RAM. These specifications make the runtime suitable for devices such as the Raspberry Pi Zero W, Cortex-M microcontrollers, and WebAssembly hosts.

### Network Architecture

The deployment model assumes air-gapped or intermittently connected devices. According to the "Offline devices" section in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) (lines 159-166), the engine never contacts the network after initial installation, making it ideal for isolated industrial controllers or remote sensors.

## The Three-Step Offline Deployment Workflow

The deployment strategy implemented in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) consists of three distinct phases that separate network-dependent preparation from offline execution.

### Step 1 – Pre-Fetch the Native Engine

On a machine with internet access, download the platform-specific inference engine using the CLI. The `needle fetch` command in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) resolves the appropriate binary for your target architecture.

```bash

# Download the engine for Linux ARM64 (e.g., Raspberry Pi 4)

needle fetch --platform-tag manylinux2014_aarch64 --out /tmp/needle_engine

```

This command caches the shared object file (e.g., `libneedle.so`) and prints the full path, typically within `~/.cache/cactus-needle/v2/`.

### Step 2 – Transfer the Model Archive

Obtain a **`.cact` archive**—either from Hugging Face, a private registry, or built locally using `needle build` via [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py). This file is completely self-contained; no additional weight files or external dependencies are required. Transfer both the engine binary and the model archive to the embedded device:

```bash
scp /tmp/needle_engine/libneedle.so user@device:/opt/needle/
scp my_needle.cact user@device:/opt/needle/

```

### Step 3 – Execute Inference

Run the model using either the Python API or the native standalone runner, depending on whether a Python interpreter is available on the target.

## Deployment Methods by Target Environment

### Python Runtime Deployment

For devices running a full Linux distribution with Python support, set the `NEEDLE2_LIB_PATH` environment variable to bypass automatic engine detection and load the pre-cached binary directly.

```python
import os
import needle

# Point to the pre-downloaded engine

os.environ["NEEDLE2_LIB_PATH"] = "/opt/needle/libneedle.so"

# Load the self-contained archive

agent = needle.Needle(weights="/opt/needle/my_needle.cact")

# Execute tool-calling inference

result = agent.run("What is the temperature in the kitchen?")
print(result["results"])

```

The `needle.Needle` class handles in-process execution without spawning external processes, keeping the memory footprint within the 28 MiB constraint.

### Native Runner Deployment (No Python)

For bare-metal or minimal Linux environments lacking a Python interpreter, use the compiled native runner included with the engine distribution:

```bash

# Make the runner executable

chmod +x /opt/needle/needle

# Execute via stdin/stdout interface

echo "What is the temperature in the kitchen?" | /opt/needle/needle \
    --weights /opt/needle/my_needle.cact

```

The runner reads prompts from standard input and returns JSON responses via standard output, enabling integration with shell scripts or C applications.

### WebAssembly and Microcontroller Targets

For WebAssembly hosts or constrained microcontrollers, compile the engine using the `wasm` or `wasm-component` platform tags. The resulting `.wasm` module exports a `needle_run` function that WASI runtimes (e.g., Wasmtime, WasmEdge) can invoke directly. When `NEEDLE2_LIB_PATH` points to a `needle.component.wasm` file, the Python API automatically selects the WASI implementation.

## Building Custom Model Archives

When deploying fine-tuned models, use the export functionality in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) to merge LoRA adapters into a single distributable archive:

```bash

# Build a .cact archive from base weights and LoRA adapters

needle build --base-model llama-3-8b --lora-adapter ./custom_lora --output kitchen_assistant.cact

```

This produces the self-contained `.cact` file referenced in the deployment steps above.

## Summary

- **Ultra-low resource footprint**: The Needle 2 engine requires only ~14 MiB storage and ~28 MiB RAM, enabling deployment on microcontrollers and single-board computers.
- **True offline operation**: Once cached via `needle fetch`, the engine in `libneedle.so` requires no network connectivity, as documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md).
- **Flexible execution modes**: Deploy using the Python API (`needle.Needle`) for rich integrations, or the native runner for Python-free environments.
- **Self-contained archives**: The `.cact` format encapsulates base models and LoRA fine-tuning weights without external dependencies, handled by [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py).
- **Cross-platform support**: Platform tags like `manylinux2014_aarch64` and `wasm-component` in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) ensure compatibility across ARM64 Linux, WebAssembly, and embedded bare-metal targets.

## Frequently Asked Questions

### How much RAM is required to run Needle 2 on an embedded device?

Needle 2 executes a full inference session in approximately **28 MiB of RAM**. The entire engine binary occupies roughly **14 MiB** of storage, making it suitable for microcontrollers and single-board computers with severe memory constraints.

### Can Needle 2 operate on devices with no internet connection?

Yes. The architecture specifically supports air-gapped deployment. Pre-fetch the native engine using `needle fetch` on a connected machine, transfer the `libneedle.so` binary and `.cact` archive to the offline device, and execute inference without any network calls. The "Offline devices" section in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) (lines 159-166) explicitly documents this workflow.

### What is the difference between the Python API and the native runner?

The **Python API** (`needle.Needle`) provides programmatic access with support for tool calling and structured outputs, loading the engine via `NEEDLE2_LIB_PATH`. The **native runner** is a compiled binary that requires no Python interpreter, communicating via stdin/stdout JSON interfaces. Both modes use the same underlying `libneedle.so` engine and maintain identical memory footprints.

### How do I deploy Needle 2 to a WebAssembly host?

Use `needle fetch --platform-tag wasm` or `wasm-component` to generate a `.wasm` module. Transfer this module along with your `.cact` archive to the target device. Invoke the exported `needle_run` function using a WASI-compliant runtime such as Wasmtime or WasmEdge. The Python package can also consume the WASM component directly when `NEEDLE2_LIB_PATH` points to the `.wasm` file.