# How to Set Up a Needle Development Environment: A Step-by-Step Guide

> Learn how to set up a Needle development environment with this step-by-step guide. Clone the repo, create a Python environment, install dependencies, and get ready for AI development.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-27

---

**Setting up a Needle development environment involves cloning the repository, creating an isolated Python environment, installing core dependencies from [`requirements.txt`](https://github.com/cactus-compute/needle/blob/main/requirements.txt), and optionally installing JAX for fine-tuning capabilities—the 14 MiB inference engine binary downloads automatically on first use.**

This guide walks you through the complete Needle development environment setup process. Needle 2 is distributed as a standard Python package in the `cactus-compute/needle` repository, with a lightweight core and optional heavy dependencies for advanced workflows like LoRA fine-tuning.

## Prerequisites

Before you begin, ensure you have:

- **Python 3.9+** installed on your system
- **Git** for cloning the repository
- Approximately **50 MB** of disk space for the Python environment (plus ~14 MiB for the auto-downloaded engine binary)

## Step 1: Clone the Needle Repository

Start by obtaining the source code from GitHub.

```bash
git clone https://github.com/cactus-compute/needle.git
cd needle

```

The repository contains the full source, including the CLI at [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py), the inference runtime at [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py), and a web playground at `needle/playground/`.

## Step 2: Create an Isolated Python Environment

Use `venv` (or `conda`) to prevent dependency conflicts with your global Python installation. This is especially important because optional dependencies like JAX can be large and platform-specific.

```bash
python3 -m venv .venv
source .venv/bin/activate   # On Windows: .venv\Scripts\activate

```

## Step 3: Install Core Dependencies

Install the lightweight runtime requirements listed in [`requirements.txt`](https://github.com/cactus-compute/needle/blob/main/requirements.txt). These include **NumPy**, **Pydantic**, and other essential libraries.

```bash
pip install -r requirements.txt

```

According to the source code, this file contains the minimal dependencies needed to run inference with the pre-compiled engine binary.

## Step 4: (Optional) Install JAX for Fine-Tuning and Export

**JAX is only required** if you plan to:
- Perform LoRA fine-tuning via [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py)
- Export or quantize models via [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)
- Build a custom engine binary

Select the appropriate installation for your hardware:

```bash
pip install "jax[cpu]"        # CPU-only

# pip install "jax[cuda]"     # NVIDIA GPU

# pip install "jax[metal]"    # Apple Silicon

```

Refer to the official JAX installation guide for platform-specific details.

## Step 5: Verify Installation—Engine Auto-Download

The first time you instantiate `needle.Needle(...)`, the **14 MiB inference engine binary** is automatically fetched from Hugging Face and cached locally. No manual intervention is required.

```bash
python -c "import needle; print('Needle version:', needle.__version__)"

```

The binary contains the compiled attention network and is cached under your home directory. Subsequent runs start instantly and consume approximately **28 MiB of RAM**.

## Step 6: Run the CLI or Playground

With your Needle development environment ready, you can:

### Launch the Command-Line Interface

```bash
needle run --query "What is the weather in Paris?" --tools '[{"name":"get_weather","description":"Get current weather for a city"}]'

```

The CLI implementation resides in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) and supports quick experimentation without writing Python scripts.

### Launch the Web Playground

```bash
cd needle/playground
pip install flask   # if not already installed

python -m flask run

```

The playground provides an interactive web UI for testing queries and tools.

## Complete Setup Script

Here's a condensed version you can copy directly:

```bash

# Clone and enter repository

git clone https://github.com/cactus-compute/needle.git && cd needle

# Create and activate virtual environment

python3 -m venv .venv && source .venv/bin/activate

# Install core dependencies

pip install -r requirements.txt

# (Optional) Install JAX for fine-tuning/export

pip install "jax[cpu]"

# Verify installation—triggers engine download

python -c "import needle; print(needle.__version__)"

# Test CLI

needle run --query "Hello, Needle!" --tools '[]'

```

## Testing Your Setup with a Simple Agent

Confirm everything works by creating a tool-equipped agent. Save this as [`test_agent.py`](https://github.com/cactus-compute/needle/blob/main/test_agent.py):

```python
from needle import Needle, tool

@tool
def get_weather(city: str) -> str:
    """Return the current weather for the given city."""
    return f"The weather in {city} is sunny."

# Engine downloads automatically on first instantiation

agent = Needle(tools=[get_weather])

response = agent.run("What's the weather like in Tokyo?")
print(response["results"])

```

Run with:

```bash
python test_agent.py

```

## Fine-Tuning and Export (JAX Required)

If you installed JAX, you can fine-tune models using [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py):

```python
from needle.model.finetune import finetune

finetune(
    base_weights="cactus-needle",
    data_path="data.jsonl",      # JSONL with prompt, output, optional system

    output_weights="my_finetuned.cact",
    epochs=5,
    batch_size=8,
    learning_rate=5e-4,
)

```

Export and quantize your fine-tuned model with [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py):

```python
from needle.model.export import export

export(
    weights="my_finetuned.cact",
    export_path="my_finetuned_exported.cact",
    quantize=True,               # Enable 2-bit quantization

)

```

See [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md) for complete parameter documentation.

## Summary

Setting up a **Needle development environment** follows this flow:

- **Clone** `cactus-compute/needle` from GitHub
- **Create** an isolated Python 3.9+ virtual environment
- **Install** core dependencies via `pip install -r requirements.txt`
- **Optionally install JAX** for fine-tuning, export, or custom engine builds
- **Verify**—the 14 MiB engine binary fetches automatically on first `Needle` instantiation
- **Run** the CLI ([`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py)) or playground (`needle/playground/`) to begin development

The design keeps the base package lightweight while delivering full on-device inference performance through the auto-managed binary.

## Frequently Asked Questions

### What is the Needle inference engine binary and why does it download automatically?

The engine binary is a **14 MiB compiled artifact** containing the optimized attention network for on-device inference. It resides on Hugging Face and downloads to your local cache the first time you create a `Needle` object. This approach keeps the PyPI package small (~hundreds of KB) while ensuring you always receive the correct platform-specific binary. The cached binary occupies ~28 MiB RAM during runtime.

### Do I need JAX to use Needle for basic inference?

**No.** JAX is only required for **LoRA fine-tuning** ([`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py)), **model export/quantization** ([`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)), or building custom engine binaries. Standard inference, tool use, and playground functionality work with the core dependencies in [`requirements.txt`](https://github.com/cactus-compute/needle/blob/main/requirements.txt) alone.

### Where are the key source files located in the Needle repository?

| Purpose | File Path |
|---------|-----------|
| Public API (`Needle`, `tool`, `extract`) | [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) |
| Command-line interface | [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) |
| Core inference loop with engine loading | [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) |
| LoRA fine-tuning implementation | [`needle/model/finetune.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/finetune.py) |
| Export and quantization utilities | [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) |
| API reference documentation | [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) |
| Fine-tuning guide | [`doc/finetuning.md`](https://github.com/cactus-compute/needle/blob/main/doc/finetuning.md) |

### How much disk space and RAM does Needle require?

The Python environment with core dependencies requires approximately **50 MB**. The engine binary adds **14 MiB** on disk (downloaded once). At runtime, the engine consumes approximately **28 MiB of RAM** for model weights and activations. Optional JAX installations vary by platform but typically range from **100 MB to several GB** depending on CPU/GPU variants.