# How to Integrate Needle 2 into Your Existing Python Project

> Integrate Needle 2 into your Python project easily. Add local AI agents with decorators, JSON communication, and offline inference loops. Get started today.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: how-to-guide
- Published: 2026-08-17

---

**Needle 2 is delivered as a pure-Python package (`cactus-needle`) that bundles a 45M-parameter inference engine, enabling you to integrate local AI agents into existing codebases using decorators, JSON-in/JSON-out communication, and fully offline-capable inference loops.**

The `cactus-compute/needle` repository provides a production-ready agentic framework that runs entirely on local hardware without external API dependencies. Because Needle 2 implements a simple contract between your Python functions and the model—JSON schema declarations in, structured JSON calls out—you can embed it into web services, data pipelines, or desktop applications with minimal architectural changes.

## Installation and Engine Setup

Needle 2 distributes the Python package separately from the compiled inference engine to support air-gapped deployments.

### Installing the Package

Add `cactus-needle` to your dependency manifest and install via pip:

```python

# requirements.txt

cactus-needle

```

```bash
pip install cactus-needle          # CPU-only install

pip install "cactus-needle[gpu]"   # CUDA acceleration

pip install "cactus-needle[metal]" # Apple Silicon GPU

```

### Fetching the Binary Engine

Download the compiled engine once to enable offline operation. The CLI command caches `libneedle.so` under `~/.cache/cactus-needle/`:

```bash
needle fetch

```

According to the source code in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py), this loader handles platform-specific wheels and custom library paths, ensuring the engine initializes without network traffic on subsequent runs.

## Core Architecture

Needle 2 exposes three logical layers that map directly to source files in the repository:

- **Engine loader** ([`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py)): Manages fetching, caching, and loading of `libneedle.so`. Handles offline-device support via the `HF_HUB_OFFLINE` and `NEEDLE_LIB_PATH` environment variables.
- **Agent core** ([`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py)): Defines the `Needle` class with `run()`, `complete()`, and `extract()` methods. Implements confidence gating, tool retrieval, and conversation state management.
- **Tool helpers** ([`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py)): Supplies the `@needle.tool` decorator and `needle.Field` constraints for converting Python functions into JSON schemas that the model can emit.

## Declaring Tools with Decorators

Integration centers on exposing your existing Python functions as tools using the decorator API. The contract is strictly JSON-in/JSON-out.

Attach `@needle.tool` to any function to generate a JSON schema automatically:

```python
import needle

@needle.tool
def get_weather(city: str):
    """Get the current weather for a city."""
    return {"city": city, "temp_c": 23, "sky": "partly cloudy"}

@needle.tool
def set_thermostat(
    temperature: int,
    mode: needle.Field(choices=["heat", "cool", "auto"]) = "auto",
):
    """Set the thermostat."""
    return {"temperature": temperature, "mode": mode}

```

As implemented in [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py), the decorator attaches the schema to `fn._needle_tool` and enforces per-argument constraints using `needle.Field`, which supports choices, ranges, and regex patterns.

## Running the Agent Loop

Once tools are declared, instantiate the `Needle` class and invoke the inference engine using either automatic or manual control.

### Automatic Execution with run()

The `run()` method handles the full agentic loop, executing function calls and feeding results back until the model returns a final response:

```python
agent = needle.Needle(tools=[get_weather, set_thermostat])

response = agent.run("Make it 21 degrees and cool the living room")
print(response["results"])

# [{'temperature': 21, 'mode': 'cool'}]

```

The agent core in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) loops until the model emits a `"type": "respond"` turn, collecting all function outputs in the `results` array. Each response includes a calibrated `confidence` score you can gate against before accepting the output.

### Manual Control with complete()

For fine-grained logging, approval workflows, or custom middleware, drive the loop manually using `complete()`:

```python
turn = agent.complete("What is the weather in Paris?")

if turn["type"] == "call":
    args = turn["function_calls"][0]["arguments"]
    result = get_weather(**args)          # Execute the suggested function

    turn = agent.complete(result)          # Feed result back to the model

print(turn)  # Final response dictionary

```

This approach lets you inspect the `confidence` field at each step and delegate low-confidence requests to fallback logic.

## Structured Data Extraction

Use the one-shot `extract()` API to parse unstructured text into Pydantic models without instantiating a full agent:

```python
from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)

print(invoice.vendor, invoice.total)   # Acme Corp 1200.0

```

## Deploying on Offline Devices

Needle 2 supports air-gapped environments through environment variables. After running `needle fetch` on a connected machine, copy the cache directory to your target device:

```bash
export HF_HUB_OFFLINE=1
export NEEDLE_LIB_PATH=/path/to/libneedle.so
python my_app.py

```

With these variables set, the engine loads from local memory and produces zero network traffic during inference, making Needle 2 suitable for secure or edge-computing scenarios.

## Loading Fine-Tuned Weights

If you have produced a LoRA-tuned `.cact` file via `needle finetune`, load it by passing the weights path to the constructor:

```python
agent = needle.Needle(
    weights="my_needle.cact",
    tools=[get_weather, set_thermostat]
)

```

The API surface remains identical; only the underlying model weights differ.

## Summary

- **Install** Needle 2 via `pip install cactus-needle` and fetch the engine binary using `needle fetch`.
- **Declare tools** using `@needle.tool` from [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py) to expose Python functions as JSON schemas with typed constraints.
- **Run inference** through methods in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) using either the automatic `run()` loop or manual `complete()` calls for custom control.
- **Validate outputs** using the `confidence` score included in every response to gate critical operations.
- **Deploy offline** by setting `HF_HUB_OFFLINE=1` and `NEEDLE_LIB_PATH` to run on air-gapped devices without network access.
- **Extract structured data** using `needle.extract()` for one-shot parsing without managing conversation state.

## Frequently Asked Questions

### Do I need an internet connection to run Needle 2 in production?

No. After the initial `needle fetch` command caches `libneedle.so` in `~/.cache/cactus-needle/`, you can set `HF_HUB_OFFLINE=1` and optionally `NEEDLE_LIB_PATH` to run entirely offline. All inference executes locally in memory with no external API calls, as handled by the engine loader in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py).

### How does Needle 2 ensure valid JSON output from tool calls?

The engine implements grammar-constrained decoding compiled from your active tool set. This guarantees syntactically valid JSON that conforms to the schemas generated by `@needle.tool`. The active context supports up to 5 tools; larger catalogs trigger a contrastive retrieval head in the inference engine to select the most relevant subset.

### Can I use Needle 2 with GPU acceleration?

Yes. Install the optional GPU dependencies using `pip install "cactus-needle[gpu]"` for CUDA support or `"cactus-needle[metal]"` for Apple Silicon. The loader in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) automatically detects available hardware and initializes the appropriate backend.

### What is the maximum number of tools I can register with the agent?

You can register an unlimited number of tools, but only 5 are active in the model's context window at once. If you provide more than 5 tools, Needle 2 uses a contrastive retrieval mechanism to dynamically select the relevant subset for each query, ensuring efficient context usage while maintaining access to large tool catalogs.