How to Create a New Project with Needle: A Complete Setup Guide

To create a new project with Needle, install the cactus-needle package, decorate Python functions with @needle.tool to define tools, instantiate the Needle agent with your tool list, and call agent.run() to execute orchestrated function calls.

Needle is a self-contained 45 M-parameter model that transforms ordinary Python functions into callable tools and extracts structured data from free-form text. This guide walks through the complete workflow for spinning up a new project using the cactus-compute/needle source code, from engine initialization to deployment.

Installation and Engine Setup

Every new project starts with a one-time fetch of the binary inference engine. When you install the Python package, Needle automatically handles the native library resolution and caching.

Install the package via pip:

pip install cactus-needle

The binary engine is downloaded once and cached at ~/.cache/cactus-needle/<engine-version>/. In needle/__init__.py, the _library_path() function resolves the shared library (libneedle.so) from an environment override, a local build, or the cache directory. The weights are then loaded via needle_load during class instantiation.

Defining Tools with the @needle.tool Decorator

Tools are the core building blocks of a Needle project. You define them as standard Python functions and expose them to the model using the @needle.tool decorator.

The decorator, implemented in needle/agent/tools.py, introspects function signatures and builds JSON schemas that the engine consumes. It supports type hints, Pydantic models, and Literal types for constrained parameters.

Here is a minimal tool definition:

import needle
from typing import Literal

@needle.tool
def set_thermostat(temp: int, mode: Literal["heat", "cool", "auto"] = "auto"):
    """Set the thermostat to a temperature and mode."""
    # Integration with device SDKs happens here

    return {"temp": temp, "mode": mode, "status": "ok"}

The _resolve() helper in needle/__init__.py walks the tools argument list, converting decorated callables, Pydantic models, or raw dictionaries into validated JSON schemas before passing them to the engine.

Creating the Agent and Running Inference

With tools defined, you instantiate the Needle class and execute queries. The agent handles tool selection, execution, and grammar-constrained decoding.

Create the agent in needle/__init__.py:

agent = needle.Needle(tools=[set_thermostat])

response = agent.run("Make it 22 degrees and cool the house")
print(response["results"])

# → [{'temp': 22, 'mode': 'cool', 'status': 'ok'}]

Key architectural behaviors to understand:

• Grammar-constrained decoding: The engine compiles a byte-level grammar from your tool schemas, guaranteeing that every output contains valid JSON.
• Tool retrieval: When you supply more than five tools, a contrastive retrieval head embeds each schema once and materializes only the top-5 matches per turn, keeping context size minimal.
• Confidence gating: Every response includes a confidence score (or None for fine-tuned weights) that you can threshold programmatically.

For single-turn completion without the execution loop, use agent.complete() instead of run().

Extracting Structured Data from Text

Needle projects can also perform one-shot structured extraction without defining explicit tools. Use the extract() helper with Pydantic models:

from pydantic import BaseModel

class Invoice(BaseModel):
    vendor: str
    total: float
    due_date: str

text = "Invoice from Acme Corp, $1,200.00, due 2026-09-01"
invoice = needle.extract(text, Invoice)
print(invoice)

# → vendor='Acme Corp' total=1200.0 due_date='2026-09-01'

This pathway bypasses the tool-execution loop and uses the same grammar-constrained decoder to populate your model fields.

Fine-Tuning and Deployment Options

For production projects requiring domain-specific behavior, you can fine-tune LoRA adapters and deploy lightweight endpoints.

Fine-tune via CLI:

needle finetune data.jsonl --epochs 5 --lora-rank 16
needle build checkpoints/needle2.pkl --lora checkpoints/needle_lora.pkl --out my_needle.cact

The training logic lives in needle/model/finetune.py, while the export utility resides in needle/model/export.py.

Deploy with the playground server:

needle playground

This starts an HTTP server defined in needle/playground/server.py for interactive testing. For embedded deployment, copy the cached engine library and your .cact archive to the target device; the runtime footprint remains under 30 MiB.

Summary

• Install once: pip install cactus-needle fetches and caches the binary engine to ~/.cache/cactus-needle/.
• Define tools: Use @needle.tool in needle/agent/tools.py to convert functions into JSON schemas.
• Instantiate: Create needle.Needle(tools=[...]) in needle/__init__.py to initialize the agent.
• Execute: Call agent.run() for full tool orchestration or complete() for single-turn inference.
• Extract: Use needle.extract() with Pydantic models for structured data parsing.
• Scale: Fine-tune adapters via needle/model/finetune.py and deploy via the playground server or embedded library.

Frequently Asked Questions

How do I install Needle for offline or air-gapped environments?

Download the wheel and dependencies using pip download cactus-needle, transfer the files to your target machine, and install with pip install --no-index --find-links . cactus-needle. The engine binary will be fetched from the local cache if present, or you can pre-seed ~/.cache/cactus-needle/<version>/ with the libneedle.so and .cact weight files.

What is the difference between agent.run() and agent.complete()?

agent.run() executes the full agent loop: it prompts the model, parses the tool call, executes your Python function, and returns the final results dictionary. agent.complete() performs a single forward pass and returns the model’s raw completion without executing any tools, useful for testing prompts or non-tool workflows.

How does Needle handle more than five tools efficiently?

When you pass more than five tools to needle.Needle(), the engine in needle/__init__.py builds a contrastive retrieval index. It embeds tool schemas once at initialization and retrieves only the top-5 most relevant embeddings for each turn, preventing context window bloat while maintaining sub-30 MiB memory usage.

Can I use Pydantic models directly instead of the @needle.tool decorator?

Yes. The _resolve() function in needle/__init__.py accepts Pydantic model classes, raw JSON schema dictionaries, or decorated callables. Pass a Pydantic model directly into the tools list, and Needle will extract the schema from model_json_schema() automatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}