How to Configure Needle for a Project: A Complete Setup Guide

Configure Needle by declaring tools with @needle.tool, setting optional system facts for context, and passing a weights path for fine‑tuned models—all through the Needle class in needle/__init__.py or matching CLI flags in needle/cli.py.

Needle is a lightweight, open‑source tool‑calling engine from Cactus Compute. Learning how to configure Needle for a project unlocks both local development and production deployments, whether you use the Python API or the command‑line interface. This guide walks through every configuration surface: tool declaration, system context, model weights, engine options, and CLI workflows.

---

Declaring Tools for Needle

Tools are the primary interface between Needle and your application logic. The Needle constructor accepts tools in three forms, normalizing all of them into JSON schemas stored in self._tools_json before passing to the native engine via needle_init.

Python Functions with @needle.tool

The most common approach. Decorate any Python function and Needle extracts the signature:

import needle
from typing import Literal, Annotated
from needle import Field

@needle.tool
def set_thermostat(
    temperature: int,
    mode: Literal["heat", "cool", "auto"] = "auto"
):
    """Set the thermostat.

    Args:
        temperature: Target temperature in Celsius.
        mode: Heating strategy.
    """
    return {"temp": temperature, "mode": mode}

@needle.tool
def send_money(
    amount: Annotated[float, Field(gt=0, le=10000, description="USD")],
    to: Annotated[str, Field(pattern=r"^@[a-z0-9_]+$", description="Recipient handle")],
    memo: Annotated[str, Field(max_length=80)] = ""
):
    """Send money to a user."""
    return {"sent": amount, "to": to, "memo": memo}

The schema generation happens in build_schema (lines 99‑106 of needle/__init__.py), which introspects type hints and Annotated/Field metadata.

Pydantic Models

Pass a Pydantic BaseModel subclass for complex, nested parameter structures. Needle converts these via pydantic_schema (also in needle/__init__.py).

Raw JSON Schema

For interoperability, pass pre‑built schemas directly:

raw_tool = {
    "name": "my_tool",
    "description": "Does something useful",
    "parameters": {"type": "object", "properties": {...}}
}
agent = needle.Needle(tools=[raw_tool])

---

Setting System Facts (Optional Context)

System facts provide environmental state that helps the model interpret relative expressions like "tomorrow" or "low battery." They are not prompts—they're semicolon‑separated key‑value pairs consumed by the engine.

Accepted keys are documented in doc/apis.md (see the System facts section, lines 36‑44):

system = "date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%"
agent = needle.Needle(tools=my_tools, system=system)

Common keys include date, locale, device, battery, and timezone. Omitting system facts defaults to a minimal context.

---

Selecting Model Weights

The weights parameter controls which checkpoint Needle loads. Three patterns are supported:

Base Model (Auto‑Download)

Omit weights entirely. Needle fetches the public checkpoint from Hugging Face automatically:

agent = needle.Needle(tools=my_tools)  # uses base model

Fine‑Tuned .cact Archive

Point to a local .cact file to override the base model. The loading logic resides in _bind (lines 58‑90 of needle/__init__.py):

agent = needle.Needle(tools=my_tools, weights="my_needle.cact")

Important trade‑off: When using fine‑tuned weights, confidence scores are disabled (response["confidence"] = None). The calibration head is not fine‑tuned alongside the policy, so reliability estimates would be misleading (see lines 58‑63 of needle/__init__.py).

---

Optional Engine Configuration

Beyond tools, system, and weights, several parameters fine‑tune runtime behavior:

Parameter	Purpose	Default
tool_index_path	Path to a persisted FAISS index for fast tool retrieval when you have 5+ tools. Enables the retrieval head described in doc/apis.md lines 46‑48	None
buffer_size	C‑level response buffer size in bytes; increase if JSON responses are truncated	65536

Environment variables also affect configuration:

• NEEDLE_LIB_PATH — Override the auto‑discovered native shared library location. Set when pre‑downloading the engine or shipping custom builds (lookup logic in needle/__init__.py lines 15‑28).
• NEEDLE_HF_REPO — Change which Hugging Face repository hosts the engine.
• HF_HUB_OFFLINE — Disable network access; requires pre‑fetched assets.

Platform‑specific library selection and fetching are handled in needle/agent/fetch.py.

---

Configuring Needle via CLI

The CLI in needle/cli.py mirrors the Python API, enabling quick iteration without writing code.

Run a Query

needle run \
  --checkpoint myorg/needle-repo/my_needle.cact \
  --query "Set the house to 22°C and cool the living room" \
  --tools tools.json \
  --max-len 512

CLI parsing for run occupies lines 22‑26 of needle/cli.py.

Fetch Native Engine (Offline Devices)

Pre‑download the engine for air‑gapped deployment:

needle fetch --platform-tag manylinux2014_aarch64

Implementation in cli.py lines 35‑42.

Download Tuned Weights

needle download org/repo/weights.cact --out ./models

Handled in cli.py lines 69‑74.

---

Complete Project Configuration Example

Here's a fully configured Needle project combining all elements:

import needle
from typing import Literal, Annotated
from needle import Field

# 1️⃣ Declare tools

@needle.tool
def set_thermostat(temperature: int, mode: Literal["heat", "cool", "auto"] = "auto"):
    """Set the thermostat."""
    return {"temp": temperature, "mode": mode}

@needle.tool
def send_money(
    amount: Annotated[float, Field(gt=0, le=10000)],
    to: Annotated[str, Field(pattern=r"^@[a-z0-9_]+$")],
    memo: str = ""
):
    """Send money to a user."""
    return {"sent": amount, "to": to, "memo": memo}

# 2️⃣ System context

system = "date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 85%"

# 3️⃣ Weights and engine options

agent = needle.Needle(
    tools=[set_thermostat, send_money],
    system=system,
    weights="my_needle.cact",           # omit for base model

    tool_index_path="tool_index.pkl",   # speed up large catalogs

    buffer_size=131072                  # double default buffer

)

# 4️⃣ Execute

response = agent.run(
    "Set the house to 22°C and cool the living room, then pay @alice $150 for groceries"
)
print(response["results"])

---

Key Source Files Reference

File	Role	Lines of Interest
needle/__init__.py	Core Needle class, tool parsing, weights loading, buffer management	15‑28 (lib path), 58‑90 (_bind), 99‑106 (schema)
needle/cli.py	CLI command definitions and argument parsing	22‑26 (run), 35‑42 (fetch), 69‑74 (download)
needle/agent/fetch.py	Engine downloading, platform detection, env‑var handling	Full file
doc/apis.md	System facts specification, retrieval head docs, confidence behavior	36‑44 (system), 46‑48 (retrieval)

---

Summary

• Tools define your application's capabilities—declare them with @needle.tool, Pydantic models, or raw JSON schema.
• System facts supply environmental context via semicolon‑separated strings; they refine how the model interprets relative terms.
• Weights select the model: omit for auto‑downloaded base, or pass a .cact path for fine‑tuned behavior (with confidence disabled).
• Engine options (tool_index_path, buffer_size, environment variables) tune performance and deployment flexibility.
• CLI parity means the same configuration works in Python or shell scripts via needle/cli.py.

---

Frequently Asked Questions

Does Needle require a GPU?

No. Needle runs on CPU by default. The native engine selects appropriate kernels for your platform during the fetch/download phase. GPU acceleration may be available in future releases; check README.md for hardware requirements.

Can I use Needle without internet access?

Yes. Set HF_HUB_OFFLINE=1 and pre‑fetch both the native engine (needle fetch) and any tuned weights (needle download). Point NEEDLE_LIB_PATH to your local engine binary if auto‑discovery fails.

Why is my response["confidence"] returning None?

Confidence scores require the base model's calibration head. When you load fine‑tuned weights, the calibration head is not fine‑tuned, so Needle disables the score to avoid misleading estimates. This is enforced in needle/__init__.py lines 58‑63.

How many tools can Needle handle efficiently?

Needle scales to hundreds of tools, but for catalogs exceeding 5 tools, enable the retrieval head by specifying tool_index_path with a pre‑built FAISS index. This avoids linear scanning and significantly reduces latency, as documented in doc/apis.md lines 46‑48.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}