How to Install Needle 2 for Tool Calling: Complete Setup Guide

Install Needle 2 for tool calling by running pip install cactus-needle, which downloads the 14MB inference engine to ~/.cache/cactus-needle/v2/ and provides the @needle.tool decorator to expose Python functions to the model.

Needle 2 is a lightweight, open-source inference engine from the cactus-compute/needle repository designed specifically for local tool calling on resource-constrained devices. This guide covers how to install Needle 2 for tool calling, configure the offline-ready engine, and implement the decorator-based API using patterns from needle/cli.py and needle/agent/tools.py.

Installing the Cactus-Needle Package

The runtime is distributed as the Python package cactus-needle on PyPI. According to the repository's README.md, the installation process downloads a pre-built engine binary once and caches it locally for permanent offline use.

Basic Installation for Inference

For pure inference and tool-calling capabilities, install the base package:

pip install cactus-needle

This command installs the pure-Python client plus the 14MB engine binaries. The engine is fetched automatically from Hugging Face (cactus-Compute/needle2) and stored in ~/.cache/cactus-needle/v2/, ensuring all future inference runs work completely offline without network access.

Installing Training Extras

To enable fine-tuning, LoRA adapters, and model export functionality, install the optional training dependencies:

pip install "cactus-needle[train]"

This variant adds JAX and GPU-specific dependencies required for the training pipeline, while the base package remains lightweight for inference-only deployments.

Understanding the Tool-Calling Architecture

The tool-calling system relies on JSON Schema generation and grammar-constrained output. As implemented in needle/agent/tools.py, the architecture uses a decorator pattern to expose Python functions to the model with automatic schema introspection.

The @needle.tool Decorator

When you decorate a function with @needle.tool, the system automatically builds a JSON Schema describing the function's name, arguments, and docstring. The decorator implementation in needle/agent/tools.py introspects type hints and default values to construct valid schemas that the model can understand.

At runtime, a needle.Needle instance receives these tool schemas and embeds them using a built-in contrastive retrieval head. This allows the model to select relevant tools even when working with large toolsets.

The Needle Agent Loop

The high-level execution flow defined in needle/cli.py follows a strict contract:

1. Schema-Constrained Generation -- The model outputs JSON strictly conforming to the compiled grammar of available tools
2. Python Execution -- The Needle.run or Needle.complete method executes the selected Python function locally
3. Result Aggregation -- Function return values are fed back into the conversation context via the results field

This loop guarantees syntactically correct tool calls while keeping execution private and local to your device.

Offline-Ready Deployment

One of Needle 2's key features is its self-contained nature. Once installed, the system requires no network connectivity to generate outputs or execute tools, making it suitable for air-gapped environments.

Caching the Engine Binary

The engine binary is downloaded once during initial setup and cached under ~/.cache/cactus-needle/v2/. You can verify or pre-fetch the cache using the CLI:

needle fetch --generation 2

Air-Gapped Installation

For devices without internet access, manually copy the cached engine file from an online machine to the target device. Then override the default search path using the environment variable as documented in doc/apis.md (see the "Offline devices" section):

export NEEDLE2_LIB_PATH=/path/to/libneedle.so

This workflow allows deployment on completely air-gapped systems while maintaining full tool-calling capabilities.

Complete Tool-Calling Examples

The following examples demonstrate the API surface exposed in needle/__init__.py, including the Needle class and tool decorator used in needle/cli.py and needle/agent/tools.py.

Basic Weather Tool Example

Create a simple tool and execute it through the high-level agent interface:

import needle

@needle.tool
def get_weather(city: str):
    """Get the current weather for a city."""
    # In a real implementation you would call an API here

    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
response = agent.run("what's it like in Lagos right now?")
print(response["results"])

# [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

Multi-Turn Conversation with Low-Level API

For finer control over the conversation loop, use the complete method to handle tool execution manually:

import json
import needle

@needle.tool
def set_lights(room: str, on: bool, brightness: int = 0):
    """Control a room's lights."""
    return {"room": room, "on": on, "brightness": brightness}

agent = needle.Needle(tools=[set_lights])

# First turn – let the model decide what to call

turn = agent.complete("dim the living room to 30")
if turn["type"] == "call":
    args = turn["function_calls"][0]["arguments"]
    result = set_lights(**args)  # Python execution

    # Feed the result back so the model can finish the conversation

    final = agent.complete(json.dumps(result))
    print(final)

This pattern exposes the raw tool-call JSON structure while maintaining the grammar constraints defined in the engine.

Summary

• Install Needle 2 for tool calling using pip install cactus-needle for inference or pip install "cactus-needle[train]" for training capabilities
• The 14MB engine binary downloads once from Hugging Face and caches in ~/.cache/cactus-needle/v2/ for permanent offline use
• Define tools using the @needle.tool decorator from needle/agent/tools.py, which auto-generates JSON Schemas from function signatures
• Execute tool calls through needle.Needle methods defined in needle/cli.py, which manage the constraint-based generation and result aggregation loop
• Deploy to air-gapped environments by copying the cached binary and setting the NEEDLE2_LIB_PATH environment variable

Frequently Asked Questions

What is the difference between cactus-needle and Needle 2?

cactus-needle is the Python package name distributed on PyPI, while Needle 2 refers to the second-generation inference engine contained within that package. The package includes the 14MB engine binary, the @needle.tool decorator system, and the high-level Needle agent API implemented in needle/cli.py.

Can I use Needle 2 for tool calling without an internet connection?

Yes. After the initial installation downloads the engine binary to ~/.cache/cactus-needle/v2/, all inference and tool execution occurs locally. For air-gapped devices, copy the cached engine file manually and set the NEEDLE2_LIB_PATH environment variable to point to the binary location, as documented in the "Offline devices" section of doc/apis.md.

How does the @needle.tool decorator generate schemas?

The decorator implementation in needle/agent/tools.py introspects the decorated function's signature, type hints, and docstring to automatically construct a JSON Schema. This schema is embedded using a contrastive retrieval head and enforced through grammar-constrained generation in the engine, ensuring the model produces syntactically valid tool calls that match the Python function signature.

What hardware requirements does Needle 2 have?

Needle 2 runs on CPUs and requires only 14MB of storage for the engine binary. The base installation (pip install cactus-needle) includes no heavy dependencies, making it suitable for tiny devices, laptops, and edge servers. GPU support is optional and only required when installing the [train] extras for fine-tuning workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}