How to Install Cactus Compute Needle: Step-by-Step Setup Guide (CPU, GPU, and Apple Silicon)

Install Cactus Compute Needle with pip install cactus-needle — the pure-Python package automatically downloads the native inference engine on first use from Hugging Face.

This guide walks through installing Needle, the lightweight open-source inference engine from cactus-compute/needle. The package supports Python 3.9+ and runs on CPU, NVIDIA GPUs via CUDA, and Apple Silicon via Metal — all from a single pip install with optional extras.

Prerequisites

Before installing, ensure you have:

• Python 3.9 or newer
• pip configured and network access to PyPI and Hugging Face

No manual binary downloads or system dependencies are required. The native engine (~14 MB) fetches automatically when first needed.

Install Cactus Compute Needle: Core Package

The default installation provides CPU-only inference suitable for desktops and servers.

pip install cactus-needle

This command appears in the quick-start section of the repository's README at lines 29-31. The wheel contains only Python code; the compiled engine arrives on demand.

GPU and Apple Silicon Acceleration

Needle accelerates inference through JAX backends. Install the appropriate extra for your hardware.

NVIDIA GPU (CUDA)

pip install "cactus-needle[gpu]"

This pulls JAX with CUDA support. See the fine-tuning section in README lines 106-116 for the exact command.

Apple Silicon (Metal)

pip install "cactus-needle[metal]"

The Metal backend enables GPU acceleration on M1/M2/M3 Macs without additional system frameworks.

First Run: Engine Download and Caching

After pip installation completes, the native engine downloads automatically. Trigger this by importing and instantiating a Needle object:

import needle

agent = needle.Needle()
print(needle.__version__)  # e.g., 2.0.10

The engine fetch logic lives in needle/agent/fetch.py (lines 15-23) and is invoked from needle/cli.py when the Needle class instantiates (lines 120-124). The binary caches under:

~/.cache/cactus-needle/<engine-version>/

Subsequent runs use the cached engine with no network traffic.

Verify Your Installation

Confirm everything works with a minimal test:

import needle

print(f"Needle version: {needle.__version__}")

# Test basic functionality

agent = needle.Needle()
result = agent.run("What is 2 + 2?")
print(result)

Successful execution confirms both the Python API and native engine are operational.

Optional: Install with Development Tools

For contributors or those extending Needle, clone the repository and install in editable mode:

git clone https://github.com/cactus-compute/needle.git
cd needle
pip install -e ".[dev]"

Architecture: What Gets Installed

Layer	Responsibility	Key Source File
Python API	Needle, tool, extract, CLI interface	needle/__init__.py, needle/cli.py
Model definition	Simple Attention Network, quantization, LoRA adapters	needle/model/architecture.py (lines 39-45)
Native engine	14 MB inference binary, fetched on demand	needle/agent/fetch.py

This three-layer design keeps initial installation under 1 MB while delivering full performance after first use.

Quick Start After Installation

With Needle installed, run the built-in playground:

needle playground

Or programmatically call tools:

import needle

@needle.tool
def get_weather(city: str):
    """Get current weather for a city."""
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
print(agent.run("What's the weather in Lagos?")["results"])

Extract structured data with Pydantic:

from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

invoice = needle.extract(
    "Invoice from Acme Corp, $1,200.00, due 2026-09-01",
    Invoice,
)
print(invoice.vendor, invoice.total)  # Acme Corp 1200.0

Troubleshooting Common Installation Issues

Engine download fails

Check connectivity to Hugging Face. The engine downloads from Cactus-Compute/needle2. Proxy configurations may need the HF_ENDPOINT environment variable.

CUDA not detected after [gpu] install

Verify NVIDIA drivers and CUDA toolkit compatibility with your JAX version. JAX maintains strict version alignment with CUDA releases.

Slow first startup expected

The initial engine download (14 MB) occurs once. Subsequent instantiation is instantaneous from local cache.

Summary

• Core install: pip install cactus-needle — works everywhere with CPU inference
• Hardware acceleration: Add [gpu] for CUDA or [metal] for Apple Silicon
• Automatic engine: Downloads ~14 MB binary from Hugging Face on first use, caches to ~/.cache/cactus-needle/
• Source verification: Installation commands documented in README.md lines 29-31 and 106-116; engine fetch implemented in needle/agent/fetch.py

Frequently Asked Questions

Does Cactus Compute Needle require manual binary installation?

No. The pip install cactus-needle command installs only Python code. The native inference engine downloads automatically from Hugging Face (Cactus-Compute/needle2) when you first create a Needle instance, as implemented in needle/agent/fetch.py.

Can I use Needle without an internet connection after installation?

Yes, after the initial engine download. The binary caches locally under ~/.cache/cactus-needle/<engine-version>/ and loads from disk on subsequent runs. No further network access is required for inference.

What Python versions does Needle support?

Needle requires Python 3.9 or newer. The package is tested on Python 3.9 through 3.12 across Linux, macOS, and Windows platforms.

How large is the complete installation?

The pip package itself is under 1 MB. The native engine adds approximately 14 MB after first run. Total disk usage remains under 20 MB — significantly smaller than typical ML inference frameworks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}