# How to Configure Needle for a Project: A Complete Setup Guide

> Configure Needle for your project with a step by step guide. Learn to declare tools, set system facts, and specify weights for fine-tuned models efficiently.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-27

---

**Configure Needle by declaring tools with `@needle.tool`, setting optional `system` facts for context, and passing a `weights` path for fine‑tuned models—all through the `Needle` class in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) or matching CLI flags in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py).**

Needle is a lightweight, open‑source tool‑calling engine from Cactus Compute. Learning how to configure Needle for a project unlocks both local development and production deployments, whether you use the Python API or the command‑line interface. This guide walks through every configuration surface: tool declaration, system context, model weights, engine options, and CLI workflows.

---

## Declaring Tools for Needle

Tools are the primary interface between Needle and your application logic. The `Needle` constructor accepts tools in three forms, normalizing all of them into JSON schemas stored in `self._tools_json` before passing to the native engine via `needle_init`.

### Python Functions with `@needle.tool`

The most common approach. Decorate any Python function and Needle extracts the signature:

```python
import needle
from typing import Literal, Annotated
from needle import Field

@needle.tool
def set_thermostat(
    temperature: int,
    mode: Literal["heat", "cool", "auto"] = "auto"
):
    """Set the thermostat.

    Args:
        temperature: Target temperature in Celsius.
        mode: Heating strategy.
    """
    return {"temp": temperature, "mode": mode}

@needle.tool
def send_money(
    amount: Annotated[float, Field(gt=0, le=10000, description="USD")],
    to: Annotated[str, Field(pattern=r"^@[a-z0-9_]+$", description="Recipient handle")],
    memo: Annotated[str, Field(max_length=80)] = ""
):
    """Send money to a user."""
    return {"sent": amount, "to": to, "memo": memo}

```

The schema generation happens in `build_schema` (lines 99‑106 of [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)), which introspects type hints and `Annotated`/`Field` metadata.

### Pydantic Models

Pass a Pydantic `BaseModel` subclass for complex, nested parameter structures. Needle converts these via `pydantic_schema` (also in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)).

### Raw JSON Schema

For interoperability, pass pre‑built schemas directly:

```python
raw_tool = {
    "name": "my_tool",
    "description": "Does something useful",
    "parameters": {"type": "object", "properties": {...}}
}
agent = needle.Needle(tools=[raw_tool])

```

---

## Setting System Facts (Optional Context)

System facts provide **environmental state** that helps the model interpret relative expressions like "tomorrow" or "low battery." They are **not** prompts—they're semicolon‑separated key‑value pairs consumed by the engine.

Accepted keys are documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) (see the *System facts* section, lines 36‑44):

```python
system = "date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%"
agent = needle.Needle(tools=my_tools, system=system)

```

Common keys include `date`, `locale`, `device`, `battery`, and `timezone`. Omitting system facts defaults to a minimal context.

---

## Selecting Model Weights

The `weights` parameter controls which checkpoint Needle loads. Three patterns are supported:

### Base Model (Auto‑Download)

Omit `weights` entirely. Needle fetches the public checkpoint from Hugging Face automatically:

```python
agent = needle.Needle(tools=my_tools)  # uses base model

```

### Fine‑Tuned `.cact` Archive

Point to a local `.cact` file to override the base model. The loading logic resides in `_bind` (lines 58‑90 of [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)):

```python
agent = needle.Needle(tools=my_tools, weights="my_needle.cact")

```

**Important trade‑off:** When using fine‑tuned weights, **confidence scores are disabled** (`response["confidence"] = None`). The calibration head is not fine‑tuned alongside the policy, so reliability estimates would be misleading (see lines 58‑63 of [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)).

---

## Optional Engine Configuration

Beyond tools, system, and weights, several parameters fine‑tune runtime behavior:

| Parameter | Purpose | Default |
|-----------|---------|---------|
| `tool_index_path` | Path to a persisted FAISS index for fast tool retrieval when you have 5+ tools. Enables the retrieval head described in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) lines 46‑48 | `None` |
| `buffer_size` | C‑level response buffer size in bytes; increase if JSON responses are truncated | 65536 |

Environment variables also affect configuration:

- **NEEDLE_LIB_PATH** — Override the auto‑discovered native shared library location. Set when pre‑downloading the engine or shipping custom builds (lookup logic in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) lines 15‑28).
- **NEEDLE_HF_REPO** — Change which Hugging Face repository hosts the engine.
- **HF_HUB_OFFLINE** — Disable network access; requires pre‑fetched assets.

Platform‑specific library selection and fetching are handled in [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py).

---

## Configuring Needle via CLI

The CLI in [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) mirrors the Python API, enabling quick iteration without writing code.

### Run a Query

```bash
needle run \
  --checkpoint myorg/needle-repo/my_needle.cact \
  --query "Set the house to 22°C and cool the living room" \
  --tools tools.json \
  --max-len 512

```

CLI parsing for `run` occupies lines 22‑26 of [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py).

### Fetch Native Engine (Offline Devices)

Pre‑download the engine for air‑gapped deployment:

```bash
needle fetch --platform-tag manylinux2014_aarch64

```

Implementation in [`cli.py`](https://github.com/cactus-compute/needle/blob/main/cli.py) lines 35‑42.

### Download Tuned Weights

```bash
needle download org/repo/weights.cact --out ./models

```

Handled in [`cli.py`](https://github.com/cactus-compute/needle/blob/main/cli.py) lines 69‑74.

---

## Complete Project Configuration Example

Here's a fully configured Needle project combining all elements:

```python
import needle
from typing import Literal, Annotated
from needle import Field

# 1️⃣ Declare tools

@needle.tool
def set_thermostat(temperature: int, mode: Literal["heat", "cool", "auto"] = "auto"):
    """Set the thermostat."""
    return {"temp": temperature, "mode": mode}

@needle.tool
def send_money(
    amount: Annotated[float, Field(gt=0, le=10000)],
    to: Annotated[str, Field(pattern=r"^@[a-z0-9_]+$")],
    memo: str = ""
):
    """Send money to a user."""
    return {"sent": amount, "to": to, "memo": memo}

# 2️⃣ System context

system = "date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 85%"

# 3️⃣ Weights and engine options

agent = needle.Needle(
    tools=[set_thermostat, send_money],
    system=system,
    weights="my_needle.cact",           # omit for base model

    tool_index_path="tool_index.pkl",   # speed up large catalogs

    buffer_size=131072                  # double default buffer

)

# 4️⃣ Execute

response = agent.run(
    "Set the house to 22°C and cool the living room, then pay @alice $150 for groceries"
)
print(response["results"])

```

---

## Key Source Files Reference

| File | Role | Lines of Interest |
|------|------|-------------------|
| [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) | Core `Needle` class, tool parsing, weights loading, buffer management | 15‑28 (lib path), 58‑90 (`_bind`), 99‑106 (schema) |
| [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) | CLI command definitions and argument parsing | 22‑26 (`run`), 35‑42 (`fetch`), 69‑74 (`download`) |
| [`needle/agent/fetch.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/fetch.py) | Engine downloading, platform detection, env‑var handling | Full file |
| [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) | System facts specification, retrieval head docs, confidence behavior | 36‑44 (system), 46‑48 (retrieval) |

---

## Summary

- **Tools** define your application's capabilities—declare them with `@needle.tool`, Pydantic models, or raw JSON schema.
- **System facts** supply environmental context via semicolon‑separated strings; they refine how the model interprets relative terms.
- **Weights** select the model: omit for auto‑downloaded base, or pass a `.cact` path for fine‑tuned behavior (with confidence disabled).
- **Engine options** (`tool_index_path`, `buffer_size`, environment variables) tune performance and deployment flexibility.
- **CLI parity** means the same configuration works in Python or shell scripts via [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py).

---

## Frequently Asked Questions

### Does Needle require a GPU?

No. Needle runs on CPU by default. The native engine selects appropriate kernels for your platform during the fetch/download phase. GPU acceleration may be available in future releases; check [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) for hardware requirements.

### Can I use Needle without internet access?

Yes. Set `HF_HUB_OFFLINE=1` and pre‑fetch both the native engine (`needle fetch`) and any tuned weights (`needle download`). Point `NEEDLE_LIB_PATH` to your local engine binary if auto‑discovery fails.

### Why is my `response["confidence"]` returning `None`?

Confidence scores require the base model's calibration head. When you load fine‑tuned `weights`, the calibration head is not fine‑tuned, so Needle disables the score to avoid misleading estimates. This is enforced in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) lines 58‑63.

### How many tools can Needle handle efficiently?

Needle scales to hundreds of tools, but for catalogs exceeding 5 tools, enable the **retrieval head** by specifying `tool_index_path` with a pre‑built FAISS index. This avoids linear scanning and significantly reduces latency, as documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md) lines 46‑48.