Deploying Needle 2 on Resource-Constrained Devices: Platforms, Requirements, and Installation

Needle 2 deploys as a single 14 MB native binary that executes full inference sessions using only ~28 MB of RAM, enabling AI workloads on smartphones, ARM-based edge devices, Windows IoT hardware, and WebAssembly runtimes without requiring network connectivity.

The cactus-compute/needle repository engineers Needle 2 specifically for resource-constrained environments where memory and storage are limited. The framework distributes platform-native engine binaries that run entirely offline, making it viable for air-gapped deployments and low-power hardware categories ranging from wearable assistants to kitchen appliances.

Supported Platforms and Architecture Coverage

Needle 2 provides pre-compiled engine builds for diverse hardware architectures through the needle fetch command, as implemented in [needle/cli.py](https://github.com/cactus-compute/needle/blob/main/needle/cli.py). The following platform tags cover the primary resource-constrained device categories:

Native CPU Targets

• macos-arm64 — Apple Silicon iPhones and iPads (via Python on iOS or macOS environments)
• linux-x86_64 — Desktop-class or edge Linux boxes including Raspberry Pi 4 and Jetson Nano
• linux-aarch64 (e.g., manylinux2014_aarch64) — ARM-based embedded Linux devices such as IoT gateways and ARM micro-servers
• win_amd64 and win_arm64 — Windows laptops and ARM-based Windows IoT devices

WebAssembly and Browser Deployment

• wasm and wasm-component — Web browsers, WebAssembly runtimes on micro-controllers, and environments executing WASM (e.g., ESP-32 with Wasm-VM)

According to the repository's API documentation in [doc/apis.md](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), selecting the correct platform tag ensures the native engine binary matches the target device's CPU architecture and operating system.

Memory Footprint and Runtime Characteristics

Resource constraints define Needle 2's architecture. As documented in the [README.md](https://github.com/cactus-compute/needle/blob/main/README.md) and implemented in [needle/model/run.py](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py):

• Binary size: Single 14 MB native library per platform
• RAM usage: Approximately 28 MB during active inference sessions
• Network requirements: Zero network traffic at inference time; the model file caches locally after a single download

Because the inference engine performs no network operations during execution, devices can operate in air-gapped environments indefinitely after initial setup.

Real-World Deployment Scenarios

The repository includes environment-specific tool sets in needle/environments/*.py demonstrating practical implementations:

• Smart-home controllers (needle/environments/smart_home.py) — Manage lights, thermostats, and door locks on low-power hubs
• Wearable assistants (needle/environments/wearable.py) — Enable speech-enabled interactions on wrist-worn devices with limited battery
• Kitchen-appliance managers (needle/environments/kitchen_appliance.py) — Control ovens or coffee makers through natural language commands

These modules illustrate how the same Needle 2 model maintains tool-calling capabilities across diverse constrained hardware categories.

Installing on Air-Gapped and Embedded Systems

Deploying Needle 2 to offline or network-restricted devices requires pre-loading the Python package and native engine binary. The installation workflow uses pip download for package transfer and needle fetch to retrieve architecture-specific binaries.

1. Download packages on a connected machine:

# Download the Python package for offline transfer

pip download cactus-needle

# Fetch the engine for your target architecture (example: aarch64 Linux)

needle fetch --platform-tag manylinux2014_aarch64 --out /opt/needle_engine

2. Install on the constrained device:

# Install from local files without network access

pip install --no-index --find-links . cactus-needle

3. Initialize the model with pre-downloaded binaries:

import needle

# Load the agent using local engine and optional tuned weights

agent = needle.Needle(
    tools=[my_tool],
    weights="my_needle.cact",          # optional tuned archive

    system="device: phone; battery: 85%"  # context about the constrained device

)

# Execute inference (no network required)

result = agent.run("what's the weather in Paris?")
print(result["results"])

WebAssembly Deployment for Browser and IoT

For environments where native binaries are impractical, the WebAssembly build enables deployment in browsers and WASM-capable micro-controllers. The following workflow downloads the WASM engine and launches a local playground server:

# Download the WebAssembly engine generation 2

needle download wasm --generation 2 --out ./wasm_build

# Start the playground server for browser-based testing

python -m needle.playground.server

This approach allows Needle 2 to execute on ESP-32 devices running Wasm-VM or within web browsers without installing Python on the target device.

Summary

• Needle 2 requires only ~28 MB RAM and a 14 MB binary to execute full AI inference sessions on resource-constrained devices.
• Supported platforms include macOS ARM64, Linux x86_64/aarch64, Windows AMD64/ARM64, and WebAssembly, covering smartphones, Raspberry Pi, IoT gateways, and browsers.
• Air-gapped deployment is fully supported by pre-caching the engine binary and model files; no network connectivity is required during inference.
• Python 3.9+ is the only runtime dependency besides the native engine binary, making integration straightforward on embedded Linux systems.
• Environment modules for smart-home, wearable, and appliance use cases demonstrate production-ready implementations on low-power hardware.

Frequently Asked Questions

How much RAM does Needle 2 require on resource-constrained devices?

Needle 2 utilizes approximately 28 MB of RAM during active inference sessions, with the core engine binary consuming 14 MB of storage. This footprint allows deployment on devices with as little as 30 MB of free RAM, such as low-power IoT gateways and wearable assistants.

Can Needle 2 run without internet connectivity?

Yes. As implemented in [needle/model/run.py](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py), the engine performs zero network operations during inference. After pre-loading the model file and native binary using needle fetch --platform-tag, the system operates entirely offline, making it suitable for air-gapped industrial and military applications.

Which embedded Linux boards are compatible with Needle 2?

The repository provides pre-compiled engines for linux-x86_64 and linux-aarch64 (including manylinux2014_aarch64 tags), supporting Raspberry Pi 4, Jetson Nano, ARM-based IoT gateways, and other embedded Linux devices. Any board capable of running Python 3.9+ and providing ~30 MB RAM can execute the framework.

Does Needle 2 support browser-based or microcontroller deployment?

Yes. The wasm and wasm-component platform targets enable deployment in web browsers via WebAssembly or on micro-controllers like ESP-32 running Wasm-VM. This allows resource-constrained devices without native Python support to execute Needle 2 models through WASM runtimes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}