How to Set Up a Needle Development Environment: A Step-by-Step Guide

Setting up a Needle development environment involves cloning the repository, creating an isolated Python environment, installing core dependencies from requirements.txt, and optionally installing JAX for fine-tuning capabilities—the 14 MiB inference engine binary downloads automatically on first use.

This guide walks you through the complete Needle development environment setup process. Needle 2 is distributed as a standard Python package in the cactus-compute/needle repository, with a lightweight core and optional heavy dependencies for advanced workflows like LoRA fine-tuning.

Prerequisites

Before you begin, ensure you have:

• Python 3.9+ installed on your system
• Git for cloning the repository
• Approximately 50 MB of disk space for the Python environment (plus ~14 MiB for the auto-downloaded engine binary)

Step 1: Clone the Needle Repository

Start by obtaining the source code from GitHub.

git clone https://github.com/cactus-compute/needle.git
cd needle

The repository contains the full source, including the CLI at needle/cli.py, the inference runtime at needle/model/run.py, and a web playground at needle/playground/.

Step 2: Create an Isolated Python Environment

Use venv (or conda) to prevent dependency conflicts with your global Python installation. This is especially important because optional dependencies like JAX can be large and platform-specific.

python3 -m venv .venv
source .venv/bin/activate   # On Windows: .venv\Scripts\activate

Step 3: Install Core Dependencies

Install the lightweight runtime requirements listed in requirements.txt. These include NumPy, Pydantic, and other essential libraries.

pip install -r requirements.txt

According to the source code, this file contains the minimal dependencies needed to run inference with the pre-compiled engine binary.

Step 4: (Optional) Install JAX for Fine-Tuning and Export

JAX is only required if you plan to:

• Perform LoRA fine-tuning via needle/model/finetune.py
• Export or quantize models via needle/model/export.py
• Build a custom engine binary

Select the appropriate installation for your hardware:

pip install "jax[cpu]"        # CPU-only

# pip install "jax[cuda]"     # NVIDIA GPU

# pip install "jax[metal]"    # Apple Silicon

Refer to the official JAX installation guide for platform-specific details.

Step 5: Verify Installation—Engine Auto-Download

The first time you instantiate needle.Needle(...), the 14 MiB inference engine binary is automatically fetched from Hugging Face and cached locally. No manual intervention is required.

python -c "import needle; print('Needle version:', needle.__version__)"

The binary contains the compiled attention network and is cached under your home directory. Subsequent runs start instantly and consume approximately 28 MiB of RAM.

Step 6: Run the CLI or Playground

With your Needle development environment ready, you can:

Launch the Command-Line Interface

needle run --query "What is the weather in Paris?" --tools '[{"name":"get_weather","description":"Get current weather for a city"}]'

The CLI implementation resides in needle/cli.py and supports quick experimentation without writing Python scripts.

Launch the Web Playground

cd needle/playground
pip install flask   # if not already installed

python -m flask run

The playground provides an interactive web UI for testing queries and tools.

Complete Setup Script

Here's a condensed version you can copy directly:

# Clone and enter repository

git clone https://github.com/cactus-compute/needle.git && cd needle

# Create and activate virtual environment

python3 -m venv .venv && source .venv/bin/activate

# Install core dependencies

pip install -r requirements.txt

# (Optional) Install JAX for fine-tuning/export

pip install "jax[cpu]"

# Verify installation—triggers engine download

python -c "import needle; print(needle.__version__)"

# Test CLI

needle run --query "Hello, Needle!" --tools '[]'

Testing Your Setup with a Simple Agent

Confirm everything works by creating a tool-equipped agent. Save this as test_agent.py:

from needle import Needle, tool

@tool
def get_weather(city: str) -> str:
    """Return the current weather for the given city."""
    return f"The weather in {city} is sunny."

# Engine downloads automatically on first instantiation

agent = Needle(tools=[get_weather])

response = agent.run("What's the weather like in Tokyo?")
print(response["results"])

Run with:

python test_agent.py

Fine-Tuning and Export (JAX Required)

If you installed JAX, you can fine-tune models using needle/model/finetune.py:

from needle.model.finetune import finetune

finetune(
    base_weights="cactus-needle",
    data_path="data.jsonl",      # JSONL with prompt, output, optional system

    output_weights="my_finetuned.cact",
    epochs=5,
    batch_size=8,
    learning_rate=5e-4,
)

Export and quantize your fine-tuned model with needle/model/export.py:

from needle.model.export import export

export(
    weights="my_finetuned.cact",
    export_path="my_finetuned_exported.cact",
    quantize=True,               # Enable 2-bit quantization

)

See doc/finetuning.md for complete parameter documentation.

Summary

Setting up a Needle development environment follows this flow:

• Clone cactus-compute/needle from GitHub
• Create an isolated Python 3.9+ virtual environment
• Install core dependencies via pip install -r requirements.txt
• Optionally install JAX for fine-tuning, export, or custom engine builds
• Verify—the 14 MiB engine binary fetches automatically on first Needle instantiation
• Run the CLI (needle/cli.py) or playground (needle/playground/) to begin development

The design keeps the base package lightweight while delivering full on-device inference performance through the auto-managed binary.

Frequently Asked Questions

What is the Needle inference engine binary and why does it download automatically?

The engine binary is a 14 MiB compiled artifact containing the optimized attention network for on-device inference. It resides on Hugging Face and downloads to your local cache the first time you create a Needle object. This approach keeps the PyPI package small (~hundreds of KB) while ensuring you always receive the correct platform-specific binary. The cached binary occupies ~28 MiB RAM during runtime.

Do I need JAX to use Needle for basic inference?

No. JAX is only required for LoRA fine-tuning (needle/model/finetune.py), model export/quantization (needle/model/export.py), or building custom engine binaries. Standard inference, tool use, and playground functionality work with the core dependencies in requirements.txt alone.

Where are the key source files located in the Needle repository?

Purpose	File Path
Public API (Needle, tool, extract)	needle/__init__.py
Command-line interface	needle/cli.py
Core inference loop with engine loading	needle/model/run.py
LoRA fine-tuning implementation	needle/model/finetune.py
Export and quantization utilities	needle/model/export.py
API reference documentation	doc/apis.md
Fine-tuning guide	doc/finetuning.md

How much disk space and RAM does Needle require?

The Python environment with core dependencies requires approximately 50 MB. The engine binary adds 14 MiB on disk (downloaded once). At runtime, the engine consumes approximately 28 MiB of RAM for model weights and activations. Optional JAX installations vary by platform but typically range from 100 MB to several GB depending on CPU/GPU variants.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}