# Primary Use Cases for the Needle 2 Foundation Model: Edge Deployment and Structured Tool Calling

> Explore Needle 2's primary use cases: edge deployment and structured tool calling. This 45M-parameter model efficiently runs on low-resource devices for powerful AI applications.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: use-cases
- Published: 2026-09-09

---

**TLDR: Needle 2 is a 45M-parameter foundation model packaged as a 14MB binary that enables grammar-constrained tool calling, schema-validated data extraction, and rapid LoRA fine-tuning on devices with only 28MB of RAM.**

Needle 2 implements a "Simple Attention Network" architecture that trades model scale for deterministic, offline-capable inference. According to the cactus-compute/needle repository, the entire engine is self-contained in a single `.cact` file, making it deployable to microcontrollers, air-gapped servers, and mobile phones without network dependencies.

## Tool-Calling Agents on Constrained and Offline Devices

Needle 2 targets **edge AI agents** that must execute structured tool calls without cloud connectivity. The model restricts output to JSON-only grammar via constrained decoding, eliminating free-text hallucinations through built-in confidence gating. With a peak memory footprint of just 28MB and a 14MB binary size, the runtime performs inference entirely within the cache of tiny devices.

The implementation in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py) exposes a `Needle` class that accepts Python functions decorated with `@needle.tool`, transforming them into JSON schemas via [`needle/agent/tools.py`](https://github.com/cactus-compute/needle/blob/main/needle/agent/tools.py). As documented in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), the engine is fetched once and cached, meaning "inference itself never touches the network" after initial load.

When using pre-built domain surfaces, developers can import ready-made environments from `needle/environments/*.py`:

```python
from needle.environments import smart_home

smart_home.agent.complete("dim the study lights to 30 percent")
smart_home.run_tests()      # sanity-check the environment suite

```

For custom agents, the decorator compiles function signatures into deterministic call formats:

```python
import needle

@needle.tool
def get_weather(city: str):
    """Get the current weather for a city."""
    return {"city": city, "temp_c": 27, "sky": "clear"}

agent = needle.Needle(tools=[get_weather])
print(agent.run("what's it like in Lagos right now?")["results"])

# → [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]

```

## Structured Data Extraction via Single-Tool Calls

Needle 2 treats **data extraction** as a specialized case of tool calling rather than a separate pipeline. By declaring a Pydantic model as the sole available tool, the model reduces extraction to a single, grammar-constrained function call that guarantees well-typed outputs. This approach eliminates post-processing regex or validation logic typically required with unstructured generation.

As stated in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md), "Extraction is not a separate mode … it is tool calling with one tool." The `needle.extract` function handles the schema wrapping automatically:

```python
from pydantic import BaseModel
import needle

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)
print(invoice.vendor, invoice.total)   # → Acme Corp 1200.0

```

## Rapid Prototyping and Domain-Specific Fine-Tuning

The third primary use case involves **customizing the model for specialized tool catalogs** without increasing deployment size. Needle 2 supports LoRA-based fine-tuning where the adapted weights are stored as a separate `.cact` archive while the base engine remains unchanged. Both the base and tuned models run on the identical binary runtime, enabling on-device customization without recompilation.

According to [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md), "Needle fine‑tunes with LoRA … the tuned model is still a single `.cact`." This allows teams to distribute fine-tuned variants for specific domains—such as medical coding or industrial sensor interpretation—while keeping the 14MB footprint constant:

```python
import needle

agent = needle.Needle(
    weights="my_needle.cact",   # a LoRA‑tuned .cact file

    tools=[...],                # your domain‑specific tools

)
agent.run("perform the task defined by my tools")

```

The [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) module provides command-line utilities (`needle finetune`, `needle download`) to manage these archives.

## Simple Attention Network Architecture

Underlying these capabilities is the **Simple Attention Network** design implemented in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py). This architecture replaces traditional feed-forward layers with Hadamard-MLP blocks, employs grouped-query attention to reduce KV-cache size, and introduces an "engram" KV memory system that periodically hashes activations into low-bit slots.

These innovations allow Needle 2 to quantize down to **2-bit precision** while preserving generation quality. The compact representation—45M parameters in 14MB—explains why the model achieves "5× to 70× smaller" size compared to alternatives while maintaining functional accuracy for tool-driven workflows.

## Summary

- **Offline tool execution**: Needle 2 runs grammar-constrained agents on 28MB RAM without network calls, using JSON-only output modes defined in [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md).
- **Type-safe extraction**: Schema validation is intrinsic to the generation process via Pydantic integration in [`needle/__init__.py`](https://github.com/cactus-compute/needle/blob/main/needle/__init__.py), not a post-processing layer.
- **Zero-bloat customization**: LoRA fine-tuning produces new `.cact` files without altering the base engine binary, enabling rapid domain adaptation via [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py).
- **Extreme compression**: The Hadamard-MLP architecture and 2-bit quantization in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) deliver foundation-model capabilities at embedded-system scale.

## Frequently Asked Questions

### What hardware can run Needle 2 inference?

Devices with approximately 28MB of available RAM can execute Needle 2 inference, including smartphones, microcontrollers, and air-gapped edge servers. The entire 45M-parameter model is self-contained in a 14MB binary that requires no external downloads during inference, as implemented in the caching logic of [`doc/apis.md`](https://github.com/cactus-compute/needle/blob/main/doc/apis.md).

### How does Needle 2 guarantee valid JSON output during tool calling?

The model uses grammar-constrained decoding that restricts the output vocabulary to valid JSON tokens only, combined with confidence gating that prevents fallback to free text. This deterministic approach is handled by the core generation loop in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), ensuring that calls to `@needle.tool` decorated functions receive parseable arguments.

### Can I fine-tune Needle 2 without increasing the deployment size?

Yes. Needle 2 utilizes LoRA (Low-Rank Adaptation) fine-tuning, which stores adapter weights in a separate `.cact` archive while reusing the original 14MB base engine. The [`needle/cli.py`](https://github.com/cactus-compute/needle/blob/main/needle/cli.py) utilities manage this process, allowing the same runtime binary to load either base or fine-tuned weights without recompilation or binary bloat.

### What makes the Simple Attention Network different from standard transformers?

The architecture replaces feed-forward layers with Hadamard-MLP blocks, uses grouped-query attention to compress the KV cache, and implements an "engram" memory system that hashes key-value states into low-bit representations. These modifications—found in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)—enable aggressive 2-bit quantization while maintaining the model's ability to perform structured generation tasks.