How to Install Needle 2: A Complete Setup Guide for the Lightweight AI Agent

Install Needle 2 by running pip install cactus-needle, then let the library automatically download the 45M-parameter weights on first use, or trigger the download manually with needle build.

Needle 2 is a compact, 45M-parameter language model shipped as the cactus-needle Python package from the cactus-compute/needle repository. The entire distribution—including the inference engine and model weights—requires only approximately 14 MB of storage and 28 MB of RAM at runtime. This guide covers the exact steps to install Needle 2, configure hardware acceleration, and obtain the pre-quantized weights.

Install the Core Package

The foundation of Needle 2 installation is the cactus-needle package available on PyPI. According to the source code in pyproject.toml (lines 2-16), this package declares all required runtime dependencies including jax, flax, and numpy, ensuring a self-contained installation.

pip install cactus-needle

After installation, the needle CLI becomes available in your environment, providing commands like needle build and needle finetune as implemented in needle/cli.py.

Install Hardware-Specific Acceleration (Optional)

While Needle 2 runs efficiently on CPU, you can enable hardware acceleration for specific platforms using optional extras defined in pyproject.toml (lines 18-21).

For CUDA-enabled GPUs:

pip install "cactus-needle[gpu]"

For Apple Silicon (Metal):

pip install "cactus-needle[metal]"

These extras install the appropriate JAX backends to leverage GPU acceleration during inference, significantly speeding up token generation on compatible hardware.

Download the Model Weights

Unlike traditional models requiring manual downloads of large .bin files, Needle 2 uses an automated caching system. The weights are stored in a proprietary .cact format—a single binary file that the C++/JIT-free runtime memory-maps directly, maintaining a constant ~28 MB RAM footprint regardless of conversation length as documented in the README (lines 9-14).

Automatic Download:

The first time you instantiate needle.Needle (defined in needle/__init__.py) or run a CLI command requiring weights, the engine automatically fetches the checkpoint from Hugging Face (Cactus-Compute/needle2) and caches it locally.

import needle

# Weights download automatically on first initialization

agent = needle.Needle()
response = agent.run("Hello, world!")

Manual Download:

To pre-fetch weights without running inference, use the CLI command implemented in needle/cli.py:

needle build

This triggers the download if the cache is empty, storing the .cact file in your local cache directory.

Using Custom Weights:

If you have fine-tuned .cact files, pass the path explicitly to the Needle class:

agent = needle.Needle(weights="path/to/my_fine_tuned.cact")

Verify the Installation

Confirm successful installation by running a simple agent with a custom tool:

import needle

@needle.tool
def get_status():
    """Return system status."""
    return {"status": "operational"}

agent = needle.Needle(tools=[get_status])
result = agent.run("Check the status")
print(result)

If the weights are not yet cached, the first execution will pause briefly to download the ~14 MB .cact file from Hugging Face before returning the structured response.

Summary

• Primary command: pip install cactus-needle installs the core package with all Python dependencies.
• Hardware extras: Append [gpu] for CUDA or [metal] for Apple Silicon to enable acceleration.
• Weight management: Weights auto-download from Hugging Face on first use; manually trigger with needle build.
• File format: Needle 2 uses compact .cact binaries rather than traditional .bin checkpoints, memory-mapping them for minimal RAM usage.
• Source files: Installation logic resides in pyproject.toml, while weight loading is handled in needle/__init__.py and CLI commands in needle/cli.py.

Frequently Asked Questions

What is the difference between cactus-needle and needle2?

cactus-needle is the PyPI package name containing the inference engine, while needle2 refers to the specific 45M-parameter model weights hosted on Hugging Face under Cactus-Compute/needle2. Installing the package (cactus-needle) enables you to download and run the needle2 model weights.

Do I need to manually download the .cact weights file?

No manual download is required. The needle.Needle class automatically downloads and caches the weights the first time you instantiate it. However, you can manually trigger this download ahead of time using the needle build command to avoid runtime delays.

Can I use Needle 2 without a GPU?

Yes. Needle 2 is designed to run efficiently on CPU-only systems with a minimal memory footprint of approximately 28 MB. The base pip install cactus-needle command requires no GPU drivers or CUDA toolkit, making it suitable for edge devices and servers without dedicated graphics hardware.

Where are the weights cached locally?

The weights are cached as a single .cact binary file in your system's user cache directory. The exact location depends on your operating system, but the needle library manages this path automatically. You can override the default location by passing an explicit weights parameter to the Needle constructor pointing to your .cact file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}