How to Integrate Needle 2 into Your Existing Python Project

Needle 2 is delivered as a pure-Python package (cactus-needle) that bundles a 45M-parameter inference engine, enabling you to integrate local AI agents into existing codebases using decorators, JSON-in/JSON-out communication, and fully offline-capable inference loops.

The cactus-compute/needle repository provides a production-ready agentic framework that runs entirely on local hardware without external API dependencies. Because Needle 2 implements a simple contract between your Python functions and the model—JSON schema declarations in, structured JSON calls out—you can embed it into web services, data pipelines, or desktop applications with minimal architectural changes.

Installation and Engine Setup

Needle 2 distributes the Python package separately from the compiled inference engine to support air-gapped deployments.

Installing the Package

Add cactus-needle to your dependency manifest and install via pip:

# requirements.txt

cactus-needle

pip install cactus-needle          # CPU-only install

pip install "cactus-needle[gpu]"   # CUDA acceleration

pip install "cactus-needle[metal]" # Apple Silicon GPU

Fetching the Binary Engine

Download the compiled engine once to enable offline operation. The CLI command caches libneedle.so under ~/.cache/cactus-needle/:

needle fetch

According to the source code in needle/model/run.py, this loader handles platform-specific wheels and custom library paths, ensuring the engine initializes without network traffic on subsequent runs.

Core Architecture

Needle 2 exposes three logical layers that map directly to source files in the repository:

• Engine loader (needle/model/run.py): Manages fetching, caching, and loading of libneedle.so. Handles offline-device support via the HF_HUB_OFFLINE and NEEDLE_LIB_PATH environment variables.
• Agent core (needle/__init__.py): Defines the Needle class with run(), complete(), and extract() methods. Implements confidence gating, tool retrieval, and conversation state management.
• Tool helpers (needle/agent/tools.py): Supplies the @needle.tool decorator and needle.Field constraints for converting Python functions into JSON schemas that the model can emit.

Declaring Tools with Decorators

Integration centers on exposing your existing Python functions as tools using the decorator API. The contract is strictly JSON-in/JSON-out.

Attach @needle.tool to any function to generate a JSON schema automatically:

import needle

@needle.tool
def get_weather(city: str):
    """Get the current weather for a city."""
    return {"city": city, "temp_c": 23, "sky": "partly cloudy"}

@needle.tool
def set_thermostat(
    temperature: int,
    mode: needle.Field(choices=["heat", "cool", "auto"]) = "auto",
):
    """Set the thermostat."""
    return {"temperature": temperature, "mode": mode}

As implemented in needle/agent/tools.py, the decorator attaches the schema to fn._needle_tool and enforces per-argument constraints using needle.Field, which supports choices, ranges, and regex patterns.

Running the Agent Loop

Once tools are declared, instantiate the Needle class and invoke the inference engine using either automatic or manual control.

Automatic Execution with run()

The run() method handles the full agentic loop, executing function calls and feeding results back until the model returns a final response:

agent = needle.Needle(tools=[get_weather, set_thermostat])

response = agent.run("Make it 21 degrees and cool the living room")
print(response["results"])

# [{'temperature': 21, 'mode': 'cool'}]

The agent core in needle/__init__.py loops until the model emits a "type": "respond" turn, collecting all function outputs in the results array. Each response includes a calibrated confidence score you can gate against before accepting the output.

Manual Control with complete()

For fine-grained logging, approval workflows, or custom middleware, drive the loop manually using complete():

turn = agent.complete("What is the weather in Paris?")

if turn["type"] == "call":
    args = turn["function_calls"][0]["arguments"]
    result = get_weather(**args)          # Execute the suggested function

    turn = agent.complete(result)          # Feed result back to the model

print(turn)  # Final response dictionary

This approach lets you inspect the confidence field at each step and delegate low-confidence requests to fallback logic.

Structured Data Extraction

Use the one-shot extract() API to parse unstructured text into Pydantic models without instantiating a full agent:

from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)

print(invoice.vendor, invoice.total)   # Acme Corp 1200.0

Deploying on Offline Devices

Needle 2 supports air-gapped environments through environment variables. After running needle fetch on a connected machine, copy the cache directory to your target device:

export HF_HUB_OFFLINE=1
export NEEDLE_LIB_PATH=/path/to/libneedle.so
python my_app.py

With these variables set, the engine loads from local memory and produces zero network traffic during inference, making Needle 2 suitable for secure or edge-computing scenarios.

Loading Fine-Tuned Weights

If you have produced a LoRA-tuned .cact file via needle finetune, load it by passing the weights path to the constructor:

agent = needle.Needle(
    weights="my_needle.cact",
    tools=[get_weather, set_thermostat]
)

The API surface remains identical; only the underlying model weights differ.

Summary

• Install Needle 2 via pip install cactus-needle and fetch the engine binary using needle fetch.
• Declare tools using @needle.tool from needle/agent/tools.py to expose Python functions as JSON schemas with typed constraints.
• Run inference through methods in needle/__init__.py using either the automatic run() loop or manual complete() calls for custom control.
• Validate outputs using the confidence score included in every response to gate critical operations.
• Deploy offline by setting HF_HUB_OFFLINE=1 and NEEDLE_LIB_PATH to run on air-gapped devices without network access.
• Extract structured data using needle.extract() for one-shot parsing without managing conversation state.

Frequently Asked Questions

Do I need an internet connection to run Needle 2 in production?

No. After the initial needle fetch command caches libneedle.so in ~/.cache/cactus-needle/, you can set HF_HUB_OFFLINE=1 and optionally NEEDLE_LIB_PATH to run entirely offline. All inference executes locally in memory with no external API calls, as handled by the engine loader in needle/model/run.py.

How does Needle 2 ensure valid JSON output from tool calls?

The engine implements grammar-constrained decoding compiled from your active tool set. This guarantees syntactically valid JSON that conforms to the schemas generated by @needle.tool. The active context supports up to 5 tools; larger catalogs trigger a contrastive retrieval head in the inference engine to select the most relevant subset.

Can I use Needle 2 with GPU acceleration?

Yes. Install the optional GPU dependencies using pip install "cactus-needle[gpu]" for CUDA support or "cactus-needle[metal]" for Apple Silicon. The loader in needle/model/run.py automatically detects available hardware and initializes the appropriate backend.

What is the maximum number of tools I can register with the agent?

You can register an unlimited number of tools, but only 5 are active in the model's context window at once. If you provide more than 5 tools, Needle 2 uses a contrastive retrieval mechanism to dynamically select the relevant subset for each query, ensuring efficient context usage while maintaining access to large tool catalogs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}